DPC++ Runtime
Runtime libraries for oneAPI DPC++
cl::sycl::detail::Scheduler Class Reference

DPC++ graph scheduler class. More...

#include <detail/scheduler/scheduler.hpp>

Collaboration diagram for cl::sycl::detail::Scheduler:

Classes

class  GraphBuilder
 Graph builder class. More...
 
class  GraphProcessor
 Graph Processor provides interfaces for enqueueing commands and their dependencies to the underlying runtime. More...
 
struct  StreamBuffers
 Stream buffers structure. More...
 

Public Member Functions

EventImplPtr addCG (std::unique_ptr< detail::CG > CommandGroup, QueueImplPtr Queue)
 Registers a command group, and adds it to the dependency graph. More...
 
EventImplPtr addCopyBack (Requirement *Req)
 Registers a command group, that copies most recent memory to the memory pointed by the requirement. More...
 
void waitForEvent (EventImplPtr Event)
 Waits for the event. More...
 
void removeMemoryObject (detail::SYCLMemObjI *MemObj)
 Removes buffer from the graph. More...
 
void cleanupFinishedCommands (EventImplPtr FinishedEvent)
 Removes finished non-leaf non-alloca commands from the subgraph (assuming that all its commands have been waited for). More...
 
EventImplPtr addHostAccessor (Requirement *Req)
 Adds nodes to the graph, that update the requirement with the pointer to the host memory. More...
 
void releaseHostAccessor (Requirement *Req)
 Unblocks operations with the memory object. More...
 
void allocateStreamBuffers (stream_impl *, size_t, size_t)
 Allocate buffers in the pool for a provided stream. More...
 
void deallocateStreamBuffers (stream_impl *)
 Deallocate all stream buffers in the pool. More...
 
QueueImplPtr getDefaultHostQueue ()
 
 Scheduler ()
 
 ~Scheduler ()
 

Static Public Member Functions

static SchedulergetInstance ()
 
static MemObjRecordgetMemObjRecord (const Requirement *const Req)
 

Protected Types

using RWLockT = std::shared_timed_mutex
 
using ReadLockT = std::shared_lock< RWLockT >
 
using WriteLockT = std::unique_lock< RWLockT >
 

Protected Member Functions

void acquireWriteLock (WriteLockT &Lock)
 Provides exclusive access to std::shared_timed_mutex object with deadlock avoidance. More...
 
void cleanupCommands (const std::vector< Command * > &Cmds)
 
void waitForRecordToFinish (MemObjRecord *Record, ReadLockT &GraphReadLock)
 This function waits on all of the graph leaves which somehow use the memory object which is represented by Record. More...
 

Static Protected Member Functions

static void enqueueLeavesOfReqUnlocked (const Requirement *const Req, std::vector< Command * > &ToCleanUp)
 

Protected Attributes

GraphBuilder MGraphBuilder
 
RWLockT MGraphLock
 
std::vector< Command * > MDeferredCleanupCommands
 
std::mutex MDeferredCleanupMutex
 
QueueImplPtr DefaultHostQueue
 
std::recursive_mutex StreamBuffersPoolMutex
 
std::unordered_map< stream_impl *, StreamBuffers * > StreamBuffersPool
 

Friends

class Command
 
class DispatchHostTask
 
class queue_impl
 
class event_impl
 
class stream_impl
 
void initStream (StreamImplPtr, QueueImplPtr)
 

Detailed Description

DPC++ graph scheduler class.

Scheduler Overview

The Scheduler is a part of DPC++ RT which ensures correct execution of command groups. To achieve this Scheduler manages acyclic dependency graph (which can have independent sub-graphs) that consists of several types of nodes that represent specific commands:

  1. Allocate memory. The command represents memory allocation operation. There can be multiple allocations for a single SYCL memory object.
  2. Release memory. The command represents memory release operation.
  3. Execute command group. The command represents Command Group (kernel) execution operation.
  4. Copy memory. The command represents memory copy operation between two memory allocations of a single memory object.

As the main input Scheduler takes a command group and returns an event representing it, so it can be waited on later. When a new command group comes, Scheduler adds one or more nodes to the graph depending on the command groups' requirements. For example, if a new command group is submitted to the SYCL context which has the latest data for all the requirements, Scheduler adds a new "Execute command group" command making it dependent on all commands affecting new command group's requirements. But if one of the requirements has no up-to-date instance in the context which the command group is submitted to, Scheduler additionally inserts copy memory command (together with allocate memory command if needed).

A simple graph looks like:

dot_inline_dotgraph_3.png

Where nodes represent commands and edges represent dependencies between them. There are three commands connected by arrows which mean that before executing second command group the first one must be executed. Also before executing the first command group memory allocation must be performed.

At some point Scheduler enqueues commands to the underlying devices. To do this, Scheduler performs topological sort to get the order in which commands should be enqueued. For example, the following graph (D depends on B and C, B and C depends on A) will be enqueued in the following order:

EventA = Enqueue(A, /*Deps=*/{});
EventB = Enqueue(B, /*Deps=*/{EventA});
EventC = Enqueue(C, /*Deps=*/{EventA});
EventD = Enqueue(D, /*Deps=*/{EventB, EventC});
dot_inline_dotgraph_4.png

Implementation details

The Scheduler is split up into two parts: graph builder and graph processor.

To build dependencies, Scheduler needs to memorize memory objects and commands that modify them.

To detect that two command groups access the same memory object and create a dependency between them, Scheduler needs to store information about the memory object.

Thread safety

To ensure thread safe execution of methods, Scheduler provides access to the graph that's guarded by a read-write mutex (analog of shared mutex from C++17).

A read-write mutex allows concurrent access to read-only operations, while write operations require exclusive access.

All the methods of GraphBuilder lock the mutex in write mode because these methods can modify the graph. Methods of GraphProcessor lock the mutex in read mode as they are not modifying the graph.

Error handling

There are two sources of errors that needs to be handled in Scheduler:

  1. errors that happen during command enqueue process
  2. the error that happend during command execution.

If an error occurs during command enqueue process, the Command::enqueue method returns the faulty command. Scheduler then reschedules the command and all dependent commands (if any).

An error with command processing can happen in underlying runtime, in this case Scheduler is notified asynchronously (using callback mechanism) what triggers rescheduling.

Definition at line 358 of file scheduler.hpp.

Member Typedef Documentation

◆ ReadLockT

using cl::sycl::detail::Scheduler::ReadLockT = std::shared_lock<RWLockT>
protected

Definition at line 453 of file scheduler.hpp.

◆ RWLockT

using cl::sycl::detail::Scheduler::RWLockT = std::shared_timed_mutex
protected

Definition at line 452 of file scheduler.hpp.

◆ WriteLockT

using cl::sycl::detail::Scheduler::WriteLockT = std::unique_lock<RWLockT>
protected

Definition at line 454 of file scheduler.hpp.

Constructor & Destructor Documentation

◆ Scheduler()

cl::sycl::detail::Scheduler::Scheduler ( )

Definition at line 384 of file scheduler.cpp.

References cl::sycl::detail::getSyclObjImpl().

◆ ~Scheduler()

cl::sycl::detail::Scheduler::~Scheduler ( )

Member Function Documentation

◆ acquireWriteLock()

void cl::sycl::detail::Scheduler::acquireWriteLock ( WriteLockT Lock)
protected

Provides exclusive access to std::shared_timed_mutex object with deadlock avoidance.

Parameters
Lockis an instance of WriteLockT, created with std::defer_lock

Definition at line 415 of file scheduler.cpp.

◆ addCG()

EventImplPtr cl::sycl::detail::Scheduler::addCG ( std::unique_ptr< detail::CG CommandGroup,
QueueImplPtr  Queue 
)

Registers a command group, and adds it to the dependency graph.

It's called by SYCL's queue.submit.

Parameters
CommandGroupis a unique_ptr to a command group to be added.
Returns
an event object to wait on for command group completion.

Definition at line 73 of file scheduler.cpp.

References cl::sycl::detail::Command::getEvent(), cl::sycl::detail::initStream(), cl::sycl::detail::Command::MDeps, cl::sycl::detail::EnqueueResultT::MResult, cl::sycl::detail::Command::MUsers, and PI_INVALID_OPERATION.

◆ addCopyBack()

EventImplPtr cl::sycl::detail::Scheduler::addCopyBack ( Requirement Req)

Registers a command group, that copies most recent memory to the memory pointed by the requirement.

Parameters
Reqis a requirement that points to the memory where data is needed.
Returns
an event object to wait on for copy finish.

Definition at line 173 of file scheduler.cpp.

References cl::sycl::detail::Command::getEvent(), cl::sycl::detail::Command::getQueue(), cl::sycl::detail::EnqueueResultT::MResult, and PI_INVALID_OPERATION.

Referenced by cl::sycl::detail::SYCLMemObjT::updateHostMemory().

◆ addHostAccessor()

EventImplPtr cl::sycl::detail::Scheduler::addHostAccessor ( Requirement Req)

Adds nodes to the graph, that update the requirement with the pointer to the host memory.

Assumes the host pointer contains the latest data. New operations with the same memory object that have side effects are blocked until releaseHostAccessor(Requirement *Req) is callled.

Parameters
Reqis the requirement to be updated.
Returns
an event which indicates when these nodes are completed and host accessor is ready for use.

Definition at line 301 of file scheduler.cpp.

References cl::sycl::detail::Command::getEvent(), cl::sycl::detail::EnqueueResultT::MResult, and PI_INVALID_OPERATION.

◆ allocateStreamBuffers()

void cl::sycl::detail::Scheduler::allocateStreamBuffers ( stream_impl Impl,
size_t  StreamBufferSize,
size_t  FlushBufferSize 
)

Allocate buffers in the pool for a provided stream.

Parameters
Implto the stream object
StreamBufferSizeof the stream buffer
FlushBufferSizeof the flush buffer for a single work item

Definition at line 370 of file scheduler.cpp.

Referenced by cl::sycl::detail::stream_impl::stream_impl().

◆ cleanupCommands()

void cl::sycl::detail::Scheduler::cleanupCommands ( const std::vector< Command * > &  Cmds)
protected

Definition at line 440 of file scheduler.cpp.

Referenced by cl::sycl::detail::DispatchHostTask::operator()().

◆ cleanupFinishedCommands()

void cl::sycl::detail::Scheduler::cleanupFinishedCommands ( EventImplPtr  FinishedEvent)

Removes finished non-leaf non-alloca commands from the subgraph (assuming that all its commands have been waited for).

See also
GraphBuilder::cleanupFinishedCommands
Parameters
FinishedEventis a cleanup candidate event.

Definition at line 233 of file scheduler.cpp.

References cl::sycl::detail::deallocateStreams().

Referenced by cl::sycl::detail::event_impl::cleanupCommand().

◆ deallocateStreamBuffers()

void cl::sycl::detail::Scheduler::deallocateStreamBuffers ( stream_impl Impl)

Deallocate all stream buffers in the pool.

Parameters
Implto the stream object

Definition at line 378 of file scheduler.cpp.

◆ enqueueLeavesOfReqUnlocked()

void cl::sycl::detail::Scheduler::enqueueLeavesOfReqUnlocked ( const Requirement *const  Req,
std::vector< Command * > &  ToCleanUp 
)
staticprotected

◆ getDefaultHostQueue()

QueueImplPtr cl::sycl::detail::Scheduler::getDefaultHostQueue ( )
inline

Definition at line 442 of file scheduler.hpp.

◆ getInstance()

◆ getMemObjRecord()

MemObjRecord * cl::sycl::detail::Scheduler::getMemObjRecord ( const Requirement *const  Req)
static

◆ releaseHostAccessor()

void cl::sycl::detail::Scheduler::releaseHostAccessor ( Requirement Req)

Unblocks operations with the memory object.

Parameters
Reqis a requirement that points to the memory object being unblocked.

Definition at line 338 of file scheduler.cpp.

References cl::sycl::detail::AccessorImplHost::MBlockedCmd, and cl::sycl::detail::Command::MEnqueueStatus.

◆ removeMemoryObject()

void cl::sycl::detail::Scheduler::removeMemoryObject ( detail::SYCLMemObjI MemObj)

Removes buffer from the graph.

The lifetime of memory object descriptor begins when the first command group that uses the memory object is submitted and ends when "removeMemoryObject(...)" method is called which means there will be no command group that uses the memory object. When removeMemoryObject is called Scheduler will enqueue and wait on all release commands associated with the memory object, which effectively guarantees that all commands accessing the memory object are complete and then the resources allocated for the memory object are freed. Then all the commands affecting the memory object are removed.

This member function is used by buffer and image.

Parameters
MemObjis a memory object that points to the buffer being removed.

Definition at line 261 of file scheduler.cpp.

References cl::sycl::detail::deallocateStreams().

Referenced by cl::sycl::detail::SYCLMemObjT::updateHostMemory().

◆ waitForEvent()

void cl::sycl::detail::Scheduler::waitForEvent ( EventImplPtr  Event)

Waits for the event.

This operation is blocking. For eager execution mode this method invokes corresponding function of device API.

Parameters
Eventis a pointer to event to wait on.

Definition at line 213 of file scheduler.cpp.

Referenced by cl::sycl::detail::event_impl::wait().

◆ waitForRecordToFinish()

void cl::sycl::detail::Scheduler::waitForRecordToFinish ( MemObjRecord Record,
ReadLockT GraphReadLock 
)
protected

This function waits on all of the graph leaves which somehow use the memory object which is represented by Record.

The function is called upon destruction of memory buffer.

Parameters
Recordmemory record to await graph leaves of to finish
GraphReadLocklocked graph read lock

GraphReadLock will be unlocked/locked as needed. Upon return from the function, GraphReadLock will be left in locked state.

Definition at line 29 of file scheduler.cpp.

References cl::sycl::detail::Command::getEvent(), cl::sycl::detail::AllocaCommandBase::getReleaseCmd(), cl::sycl::detail::MemObjRecord::MAllocaCommands, cl::sycl::detail::MemObjRecord::MReadLeaves, cl::sycl::detail::EnqueueResultT::MResult, cl::sycl::detail::MemObjRecord::MWriteLeaves, PI_INVALID_OPERATION, and cl::sycl::detail::Command::resolveReleaseDependencies().

Friends And Related Function Documentation

◆ Command

friend class Command
friend

Definition at line 775 of file scheduler.hpp.

◆ DispatchHostTask

friend class DispatchHostTask
friend

Definition at line 776 of file scheduler.hpp.

◆ event_impl

friend class event_impl
friend

Definition at line 778 of file scheduler.hpp.

◆ initStream

void initStream ( StreamImplPtr  ,
QueueImplPtr   
)
friend

Definition at line 19 of file scheduler_helpers.cpp.

◆ queue_impl

friend class queue_impl
friend

Definition at line 777 of file scheduler.hpp.

◆ stream_impl

friend class stream_impl
friend

Definition at line 810 of file scheduler.hpp.

Member Data Documentation

◆ DefaultHostQueue

QueueImplPtr cl::sycl::detail::Scheduler::DefaultHostQueue
protected

Definition at line 773 of file scheduler.hpp.

◆ MDeferredCleanupCommands

std::vector<Command *> cl::sycl::detail::Scheduler::MDeferredCleanupCommands
protected

Definition at line 770 of file scheduler.hpp.

◆ MDeferredCleanupMutex

std::mutex cl::sycl::detail::Scheduler::MDeferredCleanupMutex
protected

Definition at line 771 of file scheduler.hpp.

◆ MGraphBuilder

GraphBuilder cl::sycl::detail::Scheduler::MGraphBuilder
protected

Definition at line 767 of file scheduler.hpp.

Referenced by cl::sycl::detail::Command::processDepEvent().

◆ MGraphLock

RWLockT cl::sycl::detail::Scheduler::MGraphLock
protected

◆ StreamBuffersPool

std::unordered_map<stream_impl *, StreamBuffers *> cl::sycl::detail::Scheduler::StreamBuffersPool
protected

◆ StreamBuffersPoolMutex

std::recursive_mutex cl::sycl::detail::Scheduler::StreamBuffersPoolMutex
protected

Definition at line 814 of file scheduler.hpp.


The documentation for this class was generated from the following files: