The Wolfram Language's symbolic parallel computation architecture provides a uniquely convenient mechanism for communicating and sharing resources between parallel processes. Its foundation is a virtual shared memory model, implemented on top of WSTP-based message passing, running seamlessly on arbitrary clusters or networks of processors.
SetSharedVariable — set symbols to share values across subkernels
SetSharedFunction — set functions to share downvalues across subkernels
UnsetShared — stop sharing variables or functions
CriticalSection — synchronize concurrent access to a shared resource
DistributeDefinitions — distribute definitions to all parallel subkernels
ParallelNeeds — load the same package into all parallel subkernels
ParallelEvaluate —evaluate an expression on all parallel subkernels