Sourcing Engine

Overview

The purpose of this section is to explain how the Sourcing Engine works and processes the different information coming from your OMS, as well as the different sourcing rules that are configurable in your System Integrator (SI) Portal.

Main features

The Sourcing Engine:

  • Support direct or batch mode, which is configurable at client or order level.
  • Process wave of orders for a specific merchant.
  • Receives sourcing requests via the service bus.
  • Fetch and understand store/aggregate/source relationships.
  • Deal with order errors across the system.
  • Query orders at any time, and and maintains a record of order statuses as they occur in your OMS.

Sourcing Request Scoring in the Sourcing Engine

The sourcing process consists of the following 4 steps:

Alt Text

Batch Sorting

The Sourcing Engine can process sourcing requests in direct or batch modes. This is a configurable option at the client level, or through an specific request. Sourcing requests that are processed in a direct mode are immediately sourced. However, batch mode sourcing requests are batched together with all other sourcing requests using the same mode and processed together at specific intervals (previously configured at client level).

An ISPU order is sourced, regardless of the configuration. This speeds up the preparation process of the order and the customer pickup.

The batch mode allows the Sourcing Engine to optimize the usage of source stock across a range of requests. This ensures the priority options are more likely to be sourced successfully and minimizes overall shipments and back-orders.

The sourcing engine batch sorting is currently based on three criteria:

  • Age
  • Shipping Method
  • Request Size (line count)

To minimize splits, sort by request size first, followed by shipping method and age. To ensure orders are shipped as soon as possible, sort by age, size, and then shipping method.

This configuration is available in the configuration portal, which is not accessible externally yet. Please contact Magento to change this configuration.

Option Generation

The Sourcing Engine researches the best available sourcing option, based on the client’s restrictions, by generating all viable options. Then, the Sourcing Engine explores all the possible ways to source a specific order depending on these available sources and selects the best option.

In this way, the Sourcing Engine scores each option and retains the sourcing option with the best score. Then, the result is returned to your OMS as the sourcing response along with the other 10 best scored options for reporting purposes (an admin can see the best scored options generated for each order).

Option Scoring

The Sourcing Engine works with two different types of rules:

  • Filter Rules Strict yes/no votes on a given sourcing option providing restrictions on what a customer allows. Examples of filter rules are Max Splits and Source provides required services. All rules (filter and scoring alike) can be enabled and disabled by the client in their configuration portal. During its preparation phase, the Sourcing Engine queries the configuration portal and requests the configuration for each rule. This configuration can include rule-specific information, such as the exact limit for the maximum number of shipment filter rule. They all also contain an enabled flag, by which the client can choose to use or not use any given rule.

  • Scoring Rules evaluate the sourcing options based on customer criteria (if applicable). Each enabled rule provides a score for a given sourcing option. Scores range from 10.0 (best) to 0.0 (worst) and the set of scores across all options are scaled to this range to ensure that the best options receive 10.0 and the worst receive 0.0. To accomplish this, the Sourcing Engine scores in two phases. In the first phase, it looks at all the generated sourcing options and allows each rule to gather any meta information that needs to score. This meta information is often used to scale the scoring results to the mentioned range of 0.0 to 10.0.

Scaling is important to ensure that rules can be fairly evaluated against each other. In most cases, a customer wishes to score options across multiple rules/criteria. For instance, while grouping shipments in a single order maybe important, the customer might also wish to prioritize certain warehouses. These different rules are not equally important, in order to handle these situations, it is possible for each client to specify the importance of each rule (called weight values).These weight values are, for practical purposes, boundless meaning you really could attach a weighting of 10,000,000 to one rule and 1 to another. This weighting system is providing a lot of flexibility in controlling your sourcing decisions.

Sourcing scoring rules

Available Sourcing Rules

The following are the available Sourcing Rules:

  • Configurable maximum number of splits allowed for each order
  • Weighted rule: Minimize splits for a single order
  • Weighted rule: Prioritize sources by rankings
  • Weighted rule: Prioritize sources by stock availability
  • Weighted rule: Prioritize sources by distance
  • Weighted rule: Preference for sources in the same shipping zone as the delivery address
  • Rule Weight definition for the range of 5 values (Lowest, Low, Medium, High, Highest)
  • Prevent splits across bundled products
  • Flag allowing the engine to split the bundle after reaching a threshold of failed attempts to source the request. Note that the bundle must still be completely sourced in order to successfully source the request.
  • Prevent sourcing in case some of the lines don’t have stock, and hold back the entire order as backorder
  • Flag allowing the engine to ignore the “Prevent partial sourcing rule” after reaching a threshold of failed attempts to source the request.
  • Capabilities: Source lines to sources that provide required service
  • Capabilities: Source lines to sources that provide required shipping method (additional paramenter is available to make standard shipping to be always assumed true)
  • Filter shipping zones by defining a zone mapping and assign each source to a specific zone. Orders will be sourced only to sources that are in the same shipping zone as the defined shipping address (based on two configurable criterias state or country). In case a source is not part of any defined zone can be used as a fallback.
  • Allow for configuration of maximum orders to be sourced from a given source. Each source will define the max number of orders that can be allocated on 1 day. The rule applies to all the days of the year.

Rules that can be overwritten at order level

Force sourcing decision at the order line level

In order to force a sourcing decision, you can define the source used at the order line level. If the source_location is defined then other available sourcing options are ignored. Then, lines are forced to the defined source.

If the stock is unavailable in the source_location, the line is backordered until there is available stock for this source.

Key Value Message Description
source_location (optional) magento.sales.order_management.create When the order is created, you can pre-define the source location that will ship the items. This is available for for both a Home Delivery or In Store Pick Up (ISPU) orders (ISPU requires Ship To Store feature to be enabled)

Overwrite bundle rule

The currently configured bundle rules can be overwritten via a custom attribute provided at the bundle line level.

Key Value Message Description
custom_attributes (mom_srcrr_FilterSourceBundleFromOneSource > SHIPSTOGETHER/SHIPSIMULTANEOUSLY/SHIPSEPARATELY) magento.sales.order_management.create When the order is created, you can define the sourcing rule that applies to bundles. The custom attribute overwrites the configuration defined for such rule at the Stock Aggregate level

See our Bundles page for more information about bundle rules.

Overwrite order splitting rule

Currently configured rules to determine if an order can be split, can be overwritten via a custom attribute provided within the magento.sales.order_management.create message.

Key Value Message Description
custom_attributes (mom_srcrr_FilterNoPartialSource > SHIPSTOGETHER/SHIPSIMULTANEOUSLY/SHIPSEPARATELY) magento.sales.order_management.create When the order is created, you can define if an order can be split. The custom attribute overwrites the configuration defined for such rule at the Stock Aggregate level

Example

{
    "order": {
        "id": "001",
        "store": "STORE",
        "language": "en_US",
        "vat_country": "US",
        "origin": "WEB",
        "origin_date": "2019-01-09T10:44:54+01:00",
        "custom_attributes": [
                    {
                        "name": "mom_srcrr_FilterNoPartialSource",
                        "value": "SHIPTOGETHER"
                    }
                ],
        ...        
        "lines": [
            {
                "id": "1234",
                "line_number": 1,
                "payment_reference": "1",
                "product_name": "Test product",
                "product_type": "PHYSICAL",
                "shipping_address_reference": "johndoepers",
                "sku": "sku-1"
                ...
            }
            ...
        ]
    }
}

Configure Allocation Waves

Using the OMS configuration portal, you can define when a batch process should be run (as a maximum once per hour), and to indicate which source will be considered active in each one of the executed batches. At the moment, this configuration is only defined in UTC.

A configuration defines the sorting used to process batched orders based on:

  • Size
  • Age
  • Shipping method
  • Promised delivery date

The client can use the allocation waves in two different modes:

  • Client sourcing in batch mode provokes that all new orders in the sourcing queue, run the sourcing logic at the defined time. For these clients it is possible to define which source should be considered active in a specific wave (for example, during one wave several sources you do not want them to be considered, and you want to use other available options).
  • Clients sourcing in direct mode use the waves to process orders in the exception queue (backorders). In this case, all sources are always assumed to be active for all waves (therefore, if the configuration is overwritten at the source level it will be ignored).

The wave schedule is also used to process the backorder queue. Each wave checks that the stock for queued orders is available and in that case sources the backorders.

Allocation waves times can be configured either at the root level or at the aggregate level. At root level this implies that all existing sources associated to the aggregate are considered as available during each wave.

In order to disable a specific source for a specific time, the configuration key is removed under the specific source node by:

  1. Selecting the source in the Scope dropdown
  2. Then, navigate to the configuration: Sourcing > Active Wave Times
  3. Here, Uncheck the input field DEFAULT
  4. Finally, remove the wave by clicking the trash icon

Remove Wave

If root does not have any configured wave, but there are configured waves at the aggregate level, this means that none of the sources automatically inherit the setting, therefore you can assume that is disabled. In this case, each one of the sources has to define the exact same waves that exist for the aggregate level. In order to optimize the creation of configurations, they can be defined in the MetaSource, so it is inherited by all children sources.

Integration order flow

The following are examples for an order flow illustrating the four steps previously explained.

Alt Text

Alt Text