Sourcing Engine

The sourcing engine does many tasks to help support the flow of orders in the Order Management System (OMS).

The sourcing engine:

  • Supports direct or batch mode,which is configurable at client or order level.
  • Processes waves of orders for a specific merchant.
  • Receives sourcing requests via the service bus.
  • Fetches and understands store/aggregate/source relationships.
  • Deals with order errors across the system.
  • Queries orders at any time, and maintains a record of order statuses as they occur in your OMS.

Scoring in the sourcing engine

The sourcing process, configured at the aggregate level, consists of the following four steps:

  1. Batch sorting
  2. Option generation
  3. Option scoring
  4. Best option selection

    Sourcing request scoring in the sourcing engine

Batch sorting

The sourcing engine can process sourcing requests in direct or batch modes. This is a configurable option at the client level, or through an specific request.

Sourcing requests that are processed in direct mode are immediately sourced. Sourcing requests that are processed in batch mode are batched together with all other sourcing requests using the same mode and processed together at specific intervals (which must be configured at the client level beforehand).

An in-store pickup (ISPU) order is sourced, regardless of the configuration to speed up the preparation process of the order and customer pickup.

Batch mode allows the sourcing engine to optimize the usage of source stock across a range of requests to ensure the priority options are more likely to be sourced successfully, and to minimize overall shipments and backorders.

Batch sorting is currently based on three criteria:

  • Age
  • Shipping method
  • Request size (line count)

To minimize splits, sort by request size first, followed by shipping method and age. To ensure orders are shipped as soon as possible, sort by age, size, and then shipping method.

These definitions are enabled and configured in your System Integrator (SI) Portal, which is not yet accessible externally. Please contact Magento Support for assistance.

Option generation

The Sourcing Engine researches the best available sourcing option, based on the client’s restrictions, by generating all viable options. Then, the Sourcing Engine explores all the possible ways to source a specific order depending on these available sources and selects the best option.

In this way, the Sourcing Engine scores each option and retains the sourcing option with the best score. Then, the result is returned to your OMS as the sourcing response along with the other 10 best scored options for reporting purposes (an admin can see the best scored options generated for each order).

Option scoring

The sourcing engine works with two different types of rules:

  • Filter Rules—Strict yes/no votes on a given sourcing option provide restrictions for what a customer allows. Examples of filter rules include Max Splits and Source provides required services.

    All rules (filter and scoring alike) can be enabled and disabled by the client in their SI Portal. During its preparation phase, the Sourcing Engine queries the SI Portal and requests the configuration for each rule. This configuration can include rule-specific information, such as the exact limit for the maximum number of shipment filter rule. They all also contain an enabled flag, by which the client can choose to use or not use any given rule.

  • Scoring Rules—Scoring rules help you evaluate the sourcing options based on customer criteria (if applicable). Each enabled rule provides a score for a given sourcing option. Scores range from 10.0 (best) to 0.0 (worst) and the set of scores across all options are scaled to this range to ensure that the best options receive 10.0 and the worst receive 0.0.

    To accomplish this, the Sourcing Engine scores in two phases. In the first phase, it looks at all the generated sourcing options and allows each rule to gather any meta information that needs to score. This meta information is often used to scale the scoring results to the mentioned range of 0.0 to 10.0.

    Scaling is important to ensure that rules can be fairly evaluated against each other. In most cases, a customer wishes to score options across multiple rules/criteria. For instance, while grouping shipments in a single order may be important, the customer might also wish to prioritize certain warehouses.

    These different rules are not equally important. To handle these situations, it is possible for each client to specify the importance of each rule (called weight values). These weight values are, for practical purposes, boundles—meaning you can attach a weight of 10,000,000 to one rule and a weight of 1 to another.

    This weighting system provides flexibility in controlling your sourcing decisions!

    Sourcing scoring rules

Available sourcing rules

These are the available sourcing rules:

  • Configurable maximum number of splits allowed for each order
  • Weighted rule—Minimize splits for a single order
  • Weighted rule—Prioritize sources by rankings
  • Weighted rule—Prioritize sources by stock availability
  • Weighted rule—Prioritize sources by distance
  • Weighted rule—Preference for sources in the same shipping zone as the delivery address
  • Rule Weight definition for the range of five values (Lowest, Low, Medium, High, Highest)
  • Prevent splits across bundled products
  • Flag that allows the engine to split the bundle after reaching a threshold of failed attempts to source the request. The bundle must still be completely sourced in order to successfully source the request.
  • Prevent sourcing in case some of the lines don’t have stock, and hold back the entire order as backorder.
  • Flag that allows the engine to ignore the “Prevent partial sourcing rule” after reaching a threshold of failed attempts to source the request
  • Capabilities—Source lines to sources that provide required service
  • Capabilities—Source lines to sources that provide required shipping method (additional paramenter is available to make standard shipping to be always assumed true)
  • Filter shipping zones by defining a zone mapping and assign each source to a specific zone. Orders will be sourced only to sources that are in the same shipping zone as the defined shipping address (based on two configurable criterias state or country). If a source is not part of any defined zone it can be used as a fallback.
  • Configuration of maximum orders to be sourced from a given source. Each source will define the max number of orders that can be allocated on 1 day. The rule applies to all the days of the year.

To learn more about sourcing and scoring see Order details.

Rules that can be overwritten at order level

Some rules can be overwritten at the order level, including sourcing decisions and bundle rules.

Force sourcing decision at the order line level

To force a sourcing decision, you can define the source at the order line level. If the source_location is defined then other available sourcing options are ignored. Then, lines are forced to the defined source.

If the stock is unavailable in the source_location, the line is back-ordered until there is available stock for this source.

Key Value Message Description
source_location (optional) magento.sales.order_management.create When the order is created, you can pre-define the source location that will ship the items. This is available for for both home delivery and in-store pickup orders (ISPU requires that the Ship to Store feature is enabled).

Overwrite bundle rule

The currently configured bundle rules can be overwritten with a custom attribute provided at the bundle line level.

Key Value Message Description
custom_attributes
(mom_srcrr_FilterSourceBundleFromOneSource > SHIPSTOGETHER, SHIPSIMULTANEOUSLY, SHIPSEPARATELY)
magento.sales.order_management.create When the order is created, you can define the sourcing rule that applies to bundles. The custom attribute overwrites the configuration defined for such rule at the stock aggregate level.

See our Bundles page for more information about bundle rules.

Overwrite order splitting rule

Configured rules determine if an order can be split or overwritten with a custom attribute provided within the magento.sales.order_management.create message.

Key Value Message Description
custom_attributes
(mom_srcrr_FilterNoPartialSource > SHIPSTOGETHER, SHIPSIMULTANEOUSLY, SHIPSEPARATELY)
magento.sales.order_management.create When the order is created, you can define if an order can be split. The custom attribute overwrites the configuration defined for such rule at the stock aggregate level.

Example

{
    "order": {
        "id": "001",
        "store": "STORE",
        "language": "en_US",
        "vat_country": "US",
        "origin": "WEB",
        "origin_date": "2019-01-09T10:44:54+01:00",
        "custom_attributes": [
                    {
                        "name": "mom_srcrr_FilterNoPartialSource",
                        "value": "SHIPTOGETHER"
                    }
                ],
        ...        
        "lines": [
            {
                "id": "1234",
                "line_number": 1,
                "payment_reference": "1",
                "product_name": "Test product",
                "product_type": "PHYSICAL",
                "shipping_address_reference": "johndoepers",
                "sku": "sku-1"
                ...
            }
            ...
        ]
    }
}

Configure allocation waves

Using the OMS Admin, you can define when a batch process should be run (at a maximum once per hour), and to indicate which source will be considered active in each one of the executed batches.

This configuration is only defined in UTC.

A configuration defines the sorting used to process batched orders based on:

The client can use the allocation waves in two different modes:

  • Batch mode—Client sourcing in batch mode provokes all new orders in the sourcing queue and runs the sourcing logic at the defined time. For these clients, you can define which source should be considered active in a specific wave (for example, during one wave several sources you do not want them to be considered, and you want to use other available options).
  • Direct mode—Clients sourcing in direct mode use the waves to process orders in the exception queue (backorders). In this case, all sources are always assumed to be active for all waves (if the configuration is overwritten at the source level it will be ignored).

You can see all the completed sourcing attempts for each order and available stock in the selected source during a wave. If the order cannot be sourced, a list of reasons is displayed.

The wave schedule is also used to process the backorder queue. Each wave checks that the stock for queued orders is available and in that case sources the backorders.

Allocation wave times can be configured either at the root level or at the aggregate level. At root level this implies that all existing sources associated to the aggregate are considered as available during each wave.

To disable a specific source for a specific time and remove the the configuration under the specific source node:

  1. Selecting the source in the Scope dropdown.
  2. Navigate to the configuration, Sourcing > Active Wave Times.
  3. Deselect the DEFAULT input field.
  4. Click the trash icon to remove the wave.

    Remove Wave

If root does not have any configured wave, but there are configured waves at the aggregate level, this means that none of the sources automatically inherit the setting (it is disabled). In this case, each one of the sources must define the exact same waves that exist for the aggregate level. To optimize the creation of configurations, definte it in the MetaSource so that it is inherited by all children sources.

Integration order flow

The following are example order flows illustrating the four steps previously explained.

Alt Text

Alt Text