DPC++ Runtime
Runtime libraries for oneAPI DPC++
sycl::_V1::detail::Scheduler Class Reference

DPC++ graph scheduler class. More...

#include <detail/scheduler/scheduler.hpp>

Collaboration diagram for sycl::_V1::detail::Scheduler:


struct  ForceDeferredReleaseWrapper
class  GraphBuilder
 Graph builder class. More...
struct  GraphBuildResult
class  GraphProcessor
 Graph Processor provides interfaces for enqueueing commands and their dependencies to the underlying runtime. More...

Public Member Functions

EventImplPtr addCG (std::unique_ptr< detail::CG > CommandGroup, const QueueImplPtr &Queue)
 Registers a command group, and adds it to the dependency graph. More...
EventImplPtr addCopyBack (Requirement *Req)
 Registers a command group, that copies most recent memory to the memory pointed by the requirement. More...
void waitForEvent (const EventImplPtr &Event)
 Waits for the event. More...
bool removeMemoryObject (detail::SYCLMemObjI *MemObj, bool StrictLock=true)
 Removes buffer from the graph. More...
EventImplPtr addHostAccessor (Requirement *Req)
 Adds nodes to the graph, that update the requirement with the pointer to the host memory. More...
void releaseHostAccessor (Requirement *Req)
 Unblocks operations with the memory object. More...
QueueImplPtr getDefaultHostQueue ()
const QueueImplPtrgetDefaultHostQueue () const
void deferMemObjRelease (const std::shared_ptr< detail::SYCLMemObjI > &MemObj)
void startFusion (QueueImplPtr Queue)
void cancelFusion (QueueImplPtr Queue)
EventImplPtr completeFusion (QueueImplPtr Queue, const property_list &)
bool isInFusionMode (QueueIdT Queue)
 Scheduler ()
 ~Scheduler ()
void releaseResources ()
bool isDeferredMemObjectsEmpty ()
void enqueueCommandForCG (EventImplPtr NewEvent, std::vector< Command * > &AuxilaryCmds, BlockingT Blocking=NON_BLOCKING)

Static Public Member Functions

static SchedulergetInstance ()
static MemObjRecordgetMemObjRecord (const Requirement *const Req)

Protected Types

using RWLockT = std::shared_timed_mutex
using ReadLockT = std::shared_lock< RWLockT >
using WriteLockT = std::unique_lock< RWLockT >

Protected Member Functions

WriteLockT acquireWriteLock ()
 Provides exclusive access to std::shared_timed_mutex object with deadlock avoidance. More...
ReadLockT acquireReadLock ()
 Provides shared access to std::shared_timed_mutex object with deadlock avoidance. More...
void cleanupCommands (const std::vector< Command * > &Cmds)
void NotifyHostTaskCompletion (Command *Cmd)
void cleanupDeferredMemObjects (BlockingT Blocking)
void registerAuxiliaryResources (EventImplPtr &Event, std::vector< std::shared_ptr< const void >> Resources)
void cleanupAuxiliaryResources (BlockingT Blocking)
void waitForRecordToFinish (MemObjRecord *Record, ReadLockT &GraphReadLock)
 This function waits on all of the graph leaves which somehow use the memory object which is represented by Record. More...
bool checkLeavesCompletion (MemObjRecord *Record)

Static Protected Member Functions

static void enqueueLeavesOfReqUnlocked (const Requirement *const Req, ReadLockT &GraphReadLock, std::vector< Command * > &ToCleanUp)
static void enqueueUnblockedCommands (const std::vector< EventImplPtr > &CmdsToEnqueue, ReadLockT &GraphReadLock, std::vector< Command * > &ToCleanUp)

Protected Attributes

GraphBuilder MGraphBuilder
RWLockT MGraphLock
std::vector< Command * > MDeferredCleanupCommands
std::mutex MDeferredCleanupMutex
std::vector< std::shared_ptr< SYCLMemObjI > > MDeferredMemObjRelease
std::mutex MDeferredMemReleaseMutex
std::unordered_map< EventImplPtr, std::vector< std::shared_ptr< const void > > > MAuxiliaryResources
std::mutex MAuxiliaryResourcesMutex
QueueImplPtr DefaultHostQueue

Static Protected Attributes

static thread_local bool ForceDeferredMemObjRelease = false


class SYCLMemObjT
class Command
class DispatchHostTask
class queue_impl
class event_impl
class ::MockScheduler

Detailed Description

DPC++ graph scheduler class.

Scheduler Overview

The Scheduler is a part of DPC++ RT which ensures correct execution of command groups. To achieve this Scheduler manages acyclic dependency graph (which can have independent sub-graphs) that consists of several types of nodes that represent specific commands:

  1. Allocate memory. The command represents memory allocation operation. There can be multiple allocations for a single SYCL memory object.
  2. Release memory. The command represents memory release operation.
  3. Execute command group. The command represents Command Group (kernel) execution operation.
  4. Copy memory. The command represents memory copy operation between two memory allocations of a single memory object.

As the main input Scheduler takes a command group and returns an event representing it, so it can be waited on later. When a new command group comes, Scheduler adds one or more nodes to the graph depending on the command groups' requirements. For example, if a new command group is submitted to the SYCL context which has the latest data for all the requirements, Scheduler adds a new "Execute command group" command making it dependent on all commands affecting new command group's requirements. But if one of the requirements has no up-to-date instance in the context which the command group is submitted to, Scheduler additionally inserts copy memory command (together with allocate memory command if needed).

A simple graph looks like:


Where nodes represent commands and edges represent dependencies between them. There are three commands connected by arrows which mean that before executing second command group the first one must be executed. Also before executing the first command group memory allocation must be performed.

At some point Scheduler enqueues commands to the underlying devices. To do this, Scheduler performs topological sort to get the order in which commands should be enqueued. For example, the following graph (D depends on B and C, B and C depends on A) will be enqueued in the following order:

EventA = Enqueue(A, /*Deps=*/{});
EventB = Enqueue(B, /*Deps=*/{EventA});
EventC = Enqueue(C, /*Deps=*/{EventA});
EventD = Enqueue(D, /*Deps=*/{EventB, EventC});

Implementation details

The Scheduler is split up into two parts: graph builder and graph processor.

To build dependencies, Scheduler needs to memorize memory objects and commands that modify them.

To detect that two command groups access the same memory object and create a dependency between them, Scheduler needs to store information about the memory object.

Thread safety

To ensure thread safe execution of methods, Scheduler provides access to the graph that's guarded by a read-write mutex (analog of shared mutex from C++17).

A read-write mutex allows concurrent access to read-only operations, while write operations require exclusive access.

All the methods of GraphBuilder lock the mutex in write mode because these methods can modify the graph. Methods of GraphProcessor lock the mutex in read mode as they are not modifying the graph.

Error handling

There are two sources of errors that needs to be handled in Scheduler:

  1. errors that happen during command enqueue process
  2. the error that happend during command execution.

If an error occurs during command enqueue process, the Command::enqueue method returns the faulty command. Scheduler then reschedules the command and all dependent commands (if any).

An error with command processing can happen in underlying runtime, in this case Scheduler is notified asynchronously (using callback mechanism) what triggers rescheduling.

Definition at line 363 of file scheduler.hpp.

Member Typedef Documentation

◆ ReadLockT

using sycl::_V1::detail::Scheduler::ReadLockT = std::shared_lock<RWLockT>

Definition at line 462 of file scheduler.hpp.

◆ RWLockT

using sycl::_V1::detail::Scheduler::RWLockT = std::shared_timed_mutex

Definition at line 461 of file scheduler.hpp.

◆ WriteLockT

using sycl::_V1::detail::Scheduler::WriteLockT = std::unique_lock<RWLockT>

Definition at line 463 of file scheduler.hpp.

Constructor & Destructor Documentation

◆ Scheduler()

sycl::_V1::detail::Scheduler::Scheduler ( )

Definition at line 384 of file scheduler.cpp.

References sycl::_V1::detail::getSyclObjImpl().

◆ ~Scheduler()

sycl::_V1::detail::Scheduler::~Scheduler ( )

Definition at line 394 of file scheduler.cpp.

Member Function Documentation

◆ acquireReadLock()

ReadLockT sycl::_V1::detail::Scheduler::acquireReadLock ( )

Provides shared access to std::shared_timed_mutex object with deadlock avoidance.

Definition at line 486 of file scheduler.hpp.

◆ acquireWriteLock()

WriteLockT sycl::_V1::detail::Scheduler::acquireWriteLock ( )

Provides exclusive access to std::shared_timed_mutex object with deadlock avoidance.

Definition at line 467 of file scheduler.hpp.

◆ addCG()

EventImplPtr sycl::_V1::detail::Scheduler::addCG ( std::unique_ptr< detail::CG CommandGroup,
const QueueImplPtr Queue 

Registers a command group, and adds it to the dependency graph.

It's called by SYCL's queue.submit.

CommandGroupis a unique_ptr to a command group to be added.
an event object to wait on for command group completion.

Definition at line 90 of file scheduler.cpp.

References sycl::_V1::detail::Command::getEvent(), and sycl::_V1::detail::CGExecKernel::getStreams().

◆ addCopyBack()

EventImplPtr sycl::_V1::detail::Scheduler::addCopyBack ( Requirement Req)

Registers a command group, that copies most recent memory to the memory pointed by the requirement.

Reqis a requirement that points to the memory where data is needed.
an event object to wait on for copy finish.

Definition at line 214 of file scheduler.cpp.

References sycl::_V1::detail::Command::getEvent(), sycl::_V1::detail::Command::getQueue(), and sycl::_V1::detail::EnqueueResultT::MResult.

Referenced by sycl::_V1::detail::SYCLMemObjT::updateHostMemory().

◆ addHostAccessor()

EventImplPtr sycl::_V1::detail::Scheduler::addHostAccessor ( Requirement Req)

Adds nodes to the graph, that update the requirement with the pointer to the host memory.

Assumes the host pointer contains the latest data. New operations with the same memory object that have side effects are blocked until releaseHostAccessor(Requirement *Req) is callled.

Reqis the requirement to be updated.
an event which indicates when these nodes are completed and host accessor is ready for use.

Definition at line 294 of file scheduler.cpp.

References sycl::_V1::detail::Command::getEvent(), and sycl::_V1::detail::EnqueueResultT::MResult.

◆ cancelFusion()

void sycl::_V1::detail::Scheduler::cancelFusion ( QueueImplPtr  Queue)

◆ checkLeavesCompletion()

bool sycl::_V1::detail::Scheduler::checkLeavesCompletion ( MemObjRecord Record)

◆ cleanupAuxiliaryResources()

void sycl::_V1::detail::Scheduler::cleanupAuxiliaryResources ( BlockingT  Blocking)

Definition at line 547 of file scheduler.cpp.

References sycl::_V1::detail::BLOCKING.

◆ cleanupCommands()

void sycl::_V1::detail::Scheduler::cleanupCommands ( const std::vector< Command * > &  Cmds)

Definition at line 418 of file scheduler.cpp.

References sycl::_V1::detail::NON_BLOCKING.

◆ cleanupDeferredMemObjects()

void sycl::_V1::detail::Scheduler::cleanupDeferredMemObjects ( BlockingT  Blocking)

Definition at line 493 of file scheduler.cpp.

References sycl::_V1::detail::BLOCKING.

◆ completeFusion()

EventImplPtr sycl::_V1::detail::Scheduler::completeFusion ( QueueImplPtr  Queue,
const property_list PropList 

◆ deferMemObjRelease()

void sycl::_V1::detail::Scheduler::deferMemObjRelease ( const std::shared_ptr< detail::SYCLMemObjI > &  MemObj)

◆ enqueueCommandForCG()

void sycl::_V1::detail::Scheduler::enqueueCommandForCG ( EventImplPtr  NewEvent,
std::vector< Command * > &  AuxilaryCmds,
BlockingT  Blocking = NON_BLOCKING 

◆ enqueueLeavesOfReqUnlocked()

void sycl::_V1::detail::Scheduler::enqueueLeavesOfReqUnlocked ( const Requirement *const  Req,
ReadLockT GraphReadLock,
std::vector< Command * > &  ToCleanUp 

◆ enqueueUnblockedCommands()

void sycl::_V1::detail::Scheduler::enqueueUnblockedCommands ( const std::vector< EventImplPtr > &  CmdsToEnqueue,
ReadLockT GraphReadLock,
std::vector< Command * > &  ToCleanUp 

Definition at line 368 of file scheduler.cpp.

References sycl::_V1::detail::EnqueueResultT::MResult.

◆ getDefaultHostQueue() [1/2]

QueueImplPtr sycl::_V1::detail::Scheduler::getDefaultHostQueue ( )

Definition at line 435 of file scheduler.hpp.

◆ getDefaultHostQueue() [2/2]

const QueueImplPtr& sycl::_V1::detail::Scheduler::getDefaultHostQueue ( ) const

Definition at line 437 of file scheduler.hpp.

◆ getInstance()

◆ getMemObjRecord()

MemObjRecord * sycl::_V1::detail::Scheduler::getMemObjRecord ( const Requirement *const  Req)

◆ isDeferredMemObjectsEmpty()

bool sycl::_V1::detail::Scheduler::isDeferredMemObjectsEmpty ( )

Definition at line 488 of file scheduler.cpp.

◆ isInFusionMode()

bool sycl::_V1::detail::Scheduler::isInFusionMode ( QueueIdT  Queue)

Definition at line 592 of file scheduler.cpp.

◆ NotifyHostTaskCompletion()

◆ registerAuxiliaryResources()

void sycl::_V1::detail::Scheduler::registerAuxiliaryResources ( EventImplPtr Event,
std::vector< std::shared_ptr< const void >>  Resources 

Definition at line 541 of file scheduler.cpp.

◆ releaseHostAccessor()

void sycl::_V1::detail::Scheduler::releaseHostAccessor ( Requirement Req)

Unblocks operations with the memory object.

Reqis a requirement that points to the memory object being unblocked.

Definition at line 333 of file scheduler.cpp.

References sycl::_V1::detail::AccessorImplHost::MBlockedCmd, and sycl::_V1::detail::Command::MEnqueueStatus.

◆ releaseResources()

void sycl::_V1::detail::Scheduler::releaseResources ( )

Definition at line 396 of file scheduler.cpp.

References sycl::_V1::detail::BLOCKING.

◆ removeMemoryObject()

bool sycl::_V1::detail::Scheduler::removeMemoryObject ( detail::SYCLMemObjI MemObj,
bool  StrictLock = true 

Removes buffer from the graph.

The lifetime of memory object descriptor begins when the first command group that uses the memory object is submitted and ends when "removeMemoryObject(...)" method is called which means there will be no command group that uses the memory object. When removeMemoryObject is called Scheduler will enqueue and wait on all release commands associated with the memory object, which effectively guarantees that all commands accessing the memory object are complete and then the resources allocated for the memory object are freed. Then all the commands affecting the memory object are removed.

This member function is used by buffer and image.

MemObjis a memory object that points to the buffer being removed.
StrictLockWA, is a flag used to identify if strict read and write lock are allowed or not. Default value is always applied in buffer_impl destructor. StrictLock == false is introduced for cleanupDeferredMemObjects to avoid blocking mem object release that may lead to dead lock.
WA, true if all release action completed and we could delete memory object, false otherwise, most possible reason to receive false - fail to obtain write lock.

Definition at line 266 of file scheduler.cpp.

Referenced by sycl::_V1::detail::SYCLMemObjT::updateHostMemory().

◆ startFusion()

void sycl::_V1::detail::Scheduler::startFusion ( QueueImplPtr  Queue)

Definition at line 565 of file scheduler.cpp.

Referenced by sycl::_V1::detail::fusion_wrapper_impl::start_fusion().

◆ waitForEvent()

void sycl::_V1::detail::Scheduler::waitForEvent ( const EventImplPtr Event)

Waits for the event.

This operation is blocking. For eager execution mode this method invokes corresponding function of device API.

Eventis a pointer to event to wait on.

Definition at line 256 of file scheduler.cpp.

Referenced by sycl::_V1::detail::event_impl::wait().

◆ waitForRecordToFinish()

void sycl::_V1::detail::Scheduler::waitForRecordToFinish ( MemObjRecord Record,
ReadLockT GraphReadLock 

This function waits on all of the graph leaves which somehow use the memory object which is represented by Record.

The function is called upon destruction of memory buffer.

Recordmemory record to await graph leaves of to finish
GraphReadLocklocked graph read lock

GraphReadLock will be unlocked/locked as needed. Upon return from the function, GraphReadLock will be left in locked state.

Definition at line 40 of file scheduler.cpp.

References sycl::_V1::detail::Command::getEvent(), sycl::_V1::detail::AllocaCommandBase::getReleaseCmd(), sycl::_V1::detail::MemObjRecord::MAllocaCommands, sycl::_V1::detail::MemObjRecord::MReadLeaves, sycl::_V1::detail::EnqueueResultT::MResult, sycl::_V1::detail::MemObjRecord::MWriteLeaves, and sycl::_V1::detail::Command::resolveReleaseDependencies().

Friends And Related Function Documentation

◆ ::MockScheduler

friend class ::MockScheduler

Definition at line 892 of file scheduler.hpp.

◆ Command

friend class Command

Definition at line 888 of file scheduler.hpp.

◆ DispatchHostTask

friend class DispatchHostTask

Definition at line 889 of file scheduler.hpp.

◆ event_impl

friend class event_impl

Definition at line 891 of file scheduler.hpp.

◆ queue_impl

friend class queue_impl

Definition at line 890 of file scheduler.hpp.


friend class SYCLMemObjT

Definition at line 881 of file scheduler.hpp.

Member Data Documentation

◆ DefaultHostQueue

QueueImplPtr sycl::_V1::detail::Scheduler::DefaultHostQueue

Definition at line 871 of file scheduler.hpp.

◆ ForceDeferredMemObjRelease

thread_local bool sycl::_V1::detail::Scheduler::ForceDeferredMemObjRelease = false

Definition at line 882 of file scheduler.hpp.

Referenced by sycl::_V1::detail::SYCLMemObjT::detachMemoryObject().

◆ MAuxiliaryResources

std::unordered_map<EventImplPtr, std::vector<std::shared_ptr<const void> > > sycl::_V1::detail::Scheduler::MAuxiliaryResources

Definition at line 868 of file scheduler.hpp.

◆ MAuxiliaryResourcesMutex

std::mutex sycl::_V1::detail::Scheduler::MAuxiliaryResourcesMutex

Definition at line 869 of file scheduler.hpp.

◆ MDeferredCleanupCommands

std::vector<Command *> sycl::_V1::detail::Scheduler::MDeferredCleanupCommands

Definition at line 861 of file scheduler.hpp.

◆ MDeferredCleanupMutex

std::mutex sycl::_V1::detail::Scheduler::MDeferredCleanupMutex

Definition at line 862 of file scheduler.hpp.

◆ MDeferredMemObjRelease

std::vector<std::shared_ptr<SYCLMemObjI> > sycl::_V1::detail::Scheduler::MDeferredMemObjRelease

Definition at line 864 of file scheduler.hpp.

◆ MDeferredMemReleaseMutex

std::mutex sycl::_V1::detail::Scheduler::MDeferredMemReleaseMutex

Definition at line 865 of file scheduler.hpp.

◆ MGraphBuilder

GraphBuilder sycl::_V1::detail::Scheduler::MGraphBuilder

Definition at line 858 of file scheduler.hpp.

Referenced by sycl::_V1::detail::Command::processDepEvent().

◆ MGraphLock

RWLockT sycl::_V1::detail::Scheduler::MGraphLock

Definition at line 859 of file scheduler.hpp.

The documentation for this class was generated from the following files: