[Dynamic Selection] Customization of Backends and Policies

documentation/library_guide/dynamic_selection_api/auto_tune_policy.rst

@@ -20,42 +20,48 @@ the profiling phase periodically.
 .. code:: cpp
 
   namespace oneapi::dpl::experimental {
-
-    template<typename Backend = sycl_backend> 
-    class auto_tune_policy {
-    public:
-      // useful types
-      using resource_type = typename Backend::resource_type;
-      using wait_type = typename Backend::wait_type;
-
-      class selection_type {
+  template <typename ResourceType = sycl::queue, typename ResourceAdapter = oneapi::dpl::identity,
+          typename Backend = default_backend<ResourceType, ResourceAdapter>, typename... KeyArgs>
+    class auto_tune_policy
+      : public policy_base<auto_tune_policy<ResourceType, ResourceAdapter, Backend, KeyArgs...>,
+                           ResourceAdapter, Backend, execution_info::task_time_t>
+    {
       public:
-        auto_tune_policy<Backend> get_policy() const;
-        resource_type unwrap() const;
-      };
-
-      // constructors
-      auto_tune_policy(deferred_initialization_t);
-      auto_tune_policy(uint64_t resample_interval_in_milliseconds = 0);
-      auto_tune_policy(const std::vector<resource_type>& u,
-                       uint64_t resample_interval_in_milliseconds = 0);
-
-      // deferred initializer
-      void initialize(uint64_t resample_interval_in_milliseconds = 0);
-      void initialize(const std::vector<resource_type>& u,
-                      uint64_t resample_interval_in_milliseconds = 0);
-
-      // queries
-      auto get_resources() const;
-      auto get_submission_group();
-
-      // other implementation defined functions...
+        using resource_type = ResourceType;
+        using backend_type = Backend;
+
+        auto_tune_policy(deferred_initialization_t);
+        auto_tune_policy(uint64_t resample_interval_in_milliseconds = 0);
+        auto_tune_policy(const std::vector<ResourceType>& u, ResourceAdapter adapter = {},
+                         uint64_t resample_interval_in_milliseconds = 0);
+
+        // deferred initializer
+        void initialize(uint64_t resample_interval_in_milliseconds = 0);
+        void initialize(const std::vector<resource_type>& u, uint64_t resample_interval_in_milliseconds = 0);
+        // other implementation defined functions...
     };
-  
+
   }
-
-This policy can be used with all the dynamic selection functions, such as ``select``, ``submit``,
-and ``submit_and_wait``. It can also be used with ``policy_traits``.
+
+This policy can be used with all the dynamic selection functions, such as ``submit``,
+``submit_and_wait``, and ``try_submit``, ``get_resources``, ``get_submission_group``.
+It can also be used with ``policy_traits``.
+
+Task Identification with KeyArgs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The template parameter pack ``KeyArgs`` allows the policy to track performance for submitted jobs.
+By default (empty ``KeyArgs``), all invocations of the same function share performance history.
+When ``KeyArgs`` are specified, the policy uses both the function pointer and the specified
+arguments to create a unique key for tracking performance.
+
+**Important**: The number of ``KeyArgs`` types must exactly match the number of extra arguments
+passed to the user function beyond the resource. This requirement is enforced at compile-time.
+
+For example, ``auto_tune_policy<sycl::queue, oneapi::dpl::identity, default_backend, std::size_t>``
+will track performance separately for each distinct ``std::size_t`` argument value, useful
+when performance varies with problem size. Here, one ``KeyArg`` type corresponds to one extra
+argument passed to the function after the ``sycl::queue``.
 
 Example
 -------
@@ -128,15 +134,15 @@ to choose the best resource for the given function. A simplified, expository
 implementation of the selection algorithm follows:
 
 .. code:: cpp
-
+  //not a public function, for exposition purposes only
   template<typename Function, typename ...Args>
   selection_type auto_tune_policy::select(Function&& f, Args&&...args) {
     if (initialized_) {
       auto k = make_task_key(f, args...);
       auto tuner = get_tuner(k);
       auto offset = tuner->get_resource_to_profile();
       if (offset == use_best) {
-        return selection_type {*this, tuner->best_resource_, tuner}; 
+        return selection_type{*this, tuner->best_resource_, tuner}; 
       } else {
         auto r = resources_[offset];
         return selection{*this, r, tuner}; 
@@ -165,15 +171,15 @@ Constructors
 .. list-table:: ``auto_tune_policy`` constructors
   :widths: 50 50
   :header-rows: 1
-  
+
   * - Signature
     - Description
   * - ``auto_tune_policy(deferred_initialization_t);``
     - Defers initialization. An ``initialize`` function must be called prior to use.
   * - ``auto_tune_policy(uint64_t resample_interval_in_milliseconds = 0);``
     - Initialized to use the default set of resources. An optional resampling interval can be provided.
-  * - ``auto_tune_policy(const std::vector<resource_type>& u, uint64_t resample_interval_in_milliseconds = 0);``
-    - Overrides the default set of resources. An optional resampling interval can be provided.
+  * - ``auto_tune_policy(const std::vector<ResourceType>& u, ResourceAdapter adapter = {}, uint64_t resample_interval_in_milliseconds = 0);``
+    - Overrides the default set of resources with an optional resource adapter. An optional resampling interval can be provided.
 
 .. Note::
 
@@ -184,14 +190,14 @@ Constructors
 Deferred Initialization
 -----------------------
 
-A ``auto_tune_policy`` that was constructed with deferred initialization must be 
-initialized by calling one its ``initialize`` member functions before it can be used
+An ``auto_tune_policy`` that was constructed with deferred initialization must be
+initialized by calling one of its ``initialize`` member functions before it can be used
 to select or submit.
 
-.. list-table:: ``auto_tune_policy`` constructors
+.. list-table:: ``auto_tune_policy`` initializers
   :widths: 50 50
   :header-rows: 1
-  
+
   * - Signature
     - Description
   * - ``initialize(uint64_t resample_interval_in_milliseconds = 0);``
@@ -208,13 +214,13 @@ to select or submit.
 Queries
 -------
 
-A ``auto_tune_policy`` has ``get_resources`` and ``get_submission_group`` 
+An ``auto_tune_policy`` has ``get_resources`` and ``get_submission_group``
 member functions.
 
-.. list-table:: ``auto_tune_policy`` constructors
+.. list-table:: ``auto_tune_policy`` queries
   :widths: 50 50
   :header-rows: 1
-  
+
   * - Signature
     - Description
   * - ``std::vector<resource_type> get_resources();``
@@ -224,37 +230,6 @@ member functions.
 
 Reporting Requirements
 ----------------------
-
-If a resource returned by ``select`` is used directly without calling
-``submit`` or ``submit_and_wait``, it may be necessary to call ``report``
-to provide feedback to the policy. The ``auto_tune_policy`` tracks the
-performance of submissions on each device via callbacks that report
-the execution time. The instrumentation to report these events is included 
-in the implementations of ``submit`` and ``submit_and_wait``.  However, if you 
-use ``select`` and then submit work directly to the selected resource, it 
-is necessary to explicitly report these events.
-
-.. list-table:: ``auto_tune_policy`` reporting requirements
-  :widths: 50 50
-  :header-rows: 1
-
-  * - ``execution_info``
-    - is reporting required?
-  * - ``task_submission``
-    - No
-  * - ``task_completion``
-    - No
-  * - ``task_time``
-    - Yes
-
-In generic code, it is possible to perform compile-time checks to avoid
-reporting overheads when reporting is not needed, while still writing 
-code that will work with any policy, as demonstrated below:
-
-.. code:: cpp
-
-  auto s = select(my_policy);
-  if constexpr (report_info_v<decltype(s), execution_info::task_submission_t>)
-  {
-    s.report(execution_info::task_submission);
-  }
+A ``auto_tune_policy`` requires the ``task_time`` reporting requirement. See the
+:ref:`Execution Information <execution-information>` section for more information
+about reporting requirements.

documentation/library_guide/dynamic_selection_api/backends.rst

@@ -0,0 +1,162 @@
+Backends
+########
+
+The dynamic selection API is an experimental feature in the |onedpl_long|  
+(|onedpl_short|) that selects an *execution resource* based on a chosen  
+*selection policy*. For the policies to work with different resource types.  
+the resource management and work submission mechanics are handled  
+separately by resource-specific *backends*.
+
+A backend is responsible for:  
+
+1. **Resource Management** - storing and providing access to a set of resources  
+2. **Work Submission** - executing user functions on selected resources  
+3. **Synchronization** - waiting for submitted work to complete  
+4. **Instrumentation** - optionally reporting execution information (timing, completion)  
+
+Backends are typically not directly visible to application developers - they work  
+behind the scenes to enable policies to function.
+
+SYCL Backend
+------------
+
+A sycl backend is provided to manage ``sycl::queue`` core resources, and ``sycl::queue``
+is the default resource if none is provided or deduce-able from resource arguments.
+The SYCL backend provides:
+
+- Default initialization from available SYCL devices
+- Event-based submission and waiting
+- Optional profiling and timing instrumentation
+
+When you construct a policy without specifying resources, it uses the SYCL
+backend's default initialization:
+
+.. code:: cpp
+
+  #include <oneapi/dpl/dynamic_selection>
+
+  namespace ex = oneapi::dpl::experimental;
+
+  // Uses SYCL backend with default device initialization
+  ex::round_robin_policy<> policy;
+
+  // Explicitly specifying SYCL queues
+  std::vector<sycl::queue> my_queues = { /* ... */ };
+  ex::round_robin_policy<sycl::queue> policy2{my_queues};
+
+The SYCL backend supports the reporting for :ref:`Execution Information <execution-information>`
+of ``task_submission``, ``task_completion``, and ``task_time``.
+
+Backend Architecture
+--------------------
+
+The backend system uses a layered design to support multiple resource types:
+
+**default_backend**: When you create a policy, ``default_backend`` automatically
+determines the core resource type by applying any
+:ref:`resource adapter <resource-adapters>` to your resource type. It then 
+delegates to the appropriate ``core_resource_backend`` specialization.
+
+**core_resource_backend**: Specialized backend implementations exist for specific core
+resource types (like ``sycl::queue``). A generic implementation is provided
+for core resources without an explicitly specialized implementation which provides
+a minimal amount of functionality:
+
+- Basic resource storage and retrieval
+- Simple work submission without instrumentation (No execution information reporting)
+
+To use a custom resource type with full instrumentation support, you must create a
+``core_resource_backend`` specialization.
+
+Lazy Reporting
+--------------
+
+.. _lazy_report:
+
+For asynchronous execution, backends may use *lazy reporting* where
+:ref:`Execution Information <execution-information>` is not immediately available.
+ The SYCL backend uses lazy reporting for ``task_completion`` and ``task_time``.
+
+Policies that use execution information always call the backend's
+``lazy_report()`` function before making selections, ensuring they have
+up-to-date information about completed tasks.
+
+This is transparent to application developers - the policy handles the details.
+
+Backend Traits
+--------------
+
+Backend traits provide compile-time information about backend capabilities:
+
+.. code:: cpp
+
+  namespace oneapi::dpl::experimental
+  {
+    template<typename Backend>
+    struct backend_traits {
+
+      // True if backend has explicit wait_type
+      static constexpr bool has_wait_type_v = /* ... */;
+
+      // If has_wait_type_v is True, specific type required from user functions.
+      // If has_wait_type_v is False, void (user functions must return waitable-type)
+      using wait_type = /* ... */;
+
+      // True if backend requires lazy_report() to be called
+      static constexpr bool lazy_report_v = /* ... */;
+
+      // Scratch space type for selection handles
+      template<typename... ReportReqs>
+      using selection_scratch_t = /* ... */;
+    };
+  }
+
+These traits are primarily used by policy implementers, not application developers.
+
+Wait Type Requirements
+^^^^^^^^^^^^^^^^^^^^^^
+
+Backends specify return type requirements for user-submitted functions through the
+``wait_type`` member:
+
+**Explicit wait_type**: If a backend defines a ``wait_type`` alias (e.g., ``using wait_type = sycl::event;``),
+user functions **must** return that specific type. This is typically required when the backend
+needs to instrument or track asynchronous operations.
+
+**No explicit wait_type**: If a backend does not define a ``wait_type`` alias, user functions
+may return any *waitable-type*. A waitable-type is any type with a ``wait()`` member function
+that can be called to synchronize with the operation's completion.
+
+The SYCL backend defines ``wait_type = sycl::event``, requiring user functions to return
+``sycl::event`` for proper instrumentation and synchronization.
+
+Selection Scratch Space
+^^^^^^^^^^^^^^^^^^^^^^^
+
+.. _selection_scratch_space:
+
+Backends need storage space within selection handles to implement instrumentation.
+The ``selection_scratch_t`` trait specifies what additional data a backend requires
+based on the policy's reporting needs.
+
+When a policy tracks execution information (like task timing or completion), the backend
+needs to store temporary data with each selection. For example, the SYCL backend stores
+an extra ``sycl::event`` for start-time profiling tags when ``task_time`` reporting is requested.
+
+The backend populates and uses this scratch space during work submission and reporting.
+For policies without reporting requirements, ``selection_scratch_t<>`` is empty,
+adding no overhead.
+
+Custom Backends
+---------------
+
+For advanced use cases, you can create custom backends to support new resource
+types or provide specialized instrumentation. Custom backends are created by
+specializing ``core_resource_backend`` for your resource type. For an example of
+how to do this, look at ``core_resource_backend<sycl::queue, ...>``.
+
+See Also
+--------
+
+- :doc:`policies` - Overview of selection policies
+- :doc:`functions` - Free functions for working with backends and policies