Parallel Programming Models (recap)

Shared-memory models:
- Processes: shared main memory regions
  - fork
  - shmem family/library
- Threads: program user space → sharing lightweight
  - pthreads (POSIX)
  - OpenMP
Distributed-memory models: message passing
- MPI: a specification
- Implementations: OpenMPI, MPICH
  - API → library
  - underlying middleware
- RPC: remote procedure call
  - Stubs: remote
  - broker
- Net programming: send + receive → UDP/TCP
Data-parallel models: partitioned global address space (PGAS) → shared-memory/distributed memory
- Apache Spark
- CUDA
Hybrid models: a mix of any of the above
- e.g. MPI + OpenMP, MPI + CUDA, …

Note

stopped taking notes
i think im doomed

Connor's Notes