Sourcing Engine


The purpose of this section is to explain how the Sourcing Engine works and processes the different information coming from your OMS, as well as the different sourcing rules that are configurable in your System Integrator (SI) Portal.

Main features

The Sourcing Engine:

  • Support direct or batch mode, which is configurable at client or order level.
  • Process wave of orders for a specific merchant.
  • Receives sourcing requests via the service bus.
  • Fetch and understand store/aggregate/source relationships.
  • Deal with order errors across the system.
  • Query orders at any time, and and maintains a record of order statuses as they occur in your OMS.

Sourcing Request Scoring in the Sourcing Engine

The sourcing process consists of the following 4 steps:

Alt Text

Batch Sorting

The Sourcing Engine can process sourcing requests in direct or batch modes. This is a configurable option at the client level, or through an specific request. Sourcing requests that are processed in a direct mode are immediately sourced. However, batch mode sourcing requests are batched together with all other sourcing requests using the same mode and processed together at specific intervals (previously configured at client level).

An ISPU order is sourced, regardless of the configuration. This speeds up the preparation process of the order and the customer pickup.

The batch mode allows the Sourcing Engine to optimize the usage of source stock across a range of requests. This ensures the priority options are more likely to be sourced successfully and minimizes overall shipments and back-orders.

The sourcing engine batch sorting is currently based on three criteria:

  • Age
  • Shipping Method
  • Request Size (line count)

To minimize splits, sort by request size first, followed by shipping method and age. To ensure orders are shipped as soon as possible, sort by age, size, and then shipping method.

This configuration is available in the configuration portal, which is not accessible externally yet. Please contact Magento to change this configuration.

Option Generation

The Sourcing Engine researches the best available sourcing option, based on the client’s restrictions, by generating all viable options. Then, the Sourcing Engine explores all the possible ways to source a specific order depending on these available sources and selects the best option.

In this way, the Sourcing Engine scores each option and retains the sourcing option with the best score. Then, the result is returned to your OMS as the sourcing response along with the other 10 best scored options for reporting purposes (an admin can see the best scored options generated for each order).

Option Scoring

The Sourcing Engine works with two different types of rules:

  • Filter Rules Strict yes/no votes on a given sourcing option providing restrictions on what a customer allows. Examples of filter rules are Max Splits and Source provides required services. All rules (filter and scoring alike) can be enabled and disabled by the client in their configuration portal. During its preparation phase, the Sourcing Engine queries the configuration portal and requests the configuration for each rule. This configuration can include rule-specific information, such as the exact limit for the maximum number of shipment filter rule. They all also contain an enabled flag, by which the client can choose to use or not use any given rule.

  • Scoring Rules evaluate the sourcing options based on customer criteria (if applicable). Each enabled rule provides a score for a given sourcing option. Scores range from 10.0 (best) to 0.0 (worst) and the set of scores across all options are scaled to this range to ensure that the best options receive 10.0 and the worst receive 0.0. To accomplish this, the Sourcing Engine scores in two phases. In the first phase, it looks at all the generated sourcing options and allows each rule to gather any meta information that needs to score. This meta information is often used to scale the scoring results to the mentioned range of 0.0 to 10.0.

Scaling is important to ensure that rules can be fairly evaluated against each other. In most cases, a customer wishes to score options across multiple rules/criteria. For instance, while grouping shipments in a single order maybe important, the customer might also wish to prioritize certain warehouses. These different rules are not equally important, in order to handle these situations, it is possible for each client to specify the importance of each rule (called weight values).These weight values are, for practical purposes, boundless meaning you really could attach a weighting of 10,000,000 to one rule and 1 to another. This weighting system is providing a lot of flexibility in controlling your sourcing decisions.

This configuration is available in the configuration portal, which is not accessible externally yet. Please contact Magento to change this configuration.

Sourcing scoring rules

Available Sourcing Rules

The following are the available Sourcing Rules:

  • Configurable maximum number of splits allowed for each order
  • Weighted rule: Minimize splits for a single order
  • Weighted rule: Prioritize sources by rankings
  • Weighted rule: Prioritize sources by stock availability
  • Weighted rule: Prioritize sources by distance
  • Weighted rule: Preference for sources in the same shipping zone as the delivery address
  • Rule Weight definition for the range of 5 values (Lowest, Low, Medium, High, Highest)
  • Prevent splits across bundled products
  • Flag allowing the engine to split the bundle after reaching a threshold of failed attempts to source the request. Note that the bundle must still be completely sourced in order to successfully source the request.
  • Prevent sourcing in case some of the lines don’t have stock, and hold back the entire order as backorder
  • Flag allowing the engine to ignore the “Prevent partial sourcing rule” after reaching a threshold of failed attempts to source the request.
  • Capabilities: Source lines to sources that provide required service
  • Capabilities: Source lines to sources that provide required shipping method (additional paramenter is available to make standard shipping to be always assumed true)
  • Filter shipping zones by defining a zone mapping and assign each source to a specific zone. Orders will be sourced only to sources that are in the same shipping zone as the defined shipping address (based on two configurable criterias state or country). In case a source is not part of any defined zone can be used as a fallback.
  • Allow for configuration of maximum orders to be sourced from a given source. Each source will define the max number of orders that can be allocated on 1 day. The rule applies to all the days of the year.

Configure Allocation Waves

Instead of configure allocation waves through your configuration portal for a client with a Batch mode configuration type. Depending on the defined rules, is possible to improve the way to source orders. As mentioned, using your configuration portal, you can define when a batch process should be run (only once per hour), and to indicate which source will be considered active in each one of the executed batches. At the moment, this configuration is only defined in UTC.

A source inherits its wave schedule, but can override it in order to add or remove waves. A source with no times selected becomes inactive. The wave schedule is also used to process the backorder queue. Each wave validates if the stock for the queued orders is available and in that case sources the backorders.

A configuration defines the sorting used to process batched orders based on:

  • Size
  • Age
  • Shipping method

As mentioned earlier in the Batch Sorting section, clients can use allocation waves in two different modes:

  • Client sourcing in batch mode provokes that all new orders in the exception queue, run the sourcing logic at the defined time. For these clients it is possible to define which source should be considered active in a specific wave (for example, during one wave several sources you do not want them to be considered, and you want to use other available options).
  • Clients sourcing in direct mode use the waves to process orders in the exception queue. In this case, all sources are always assumed to be active for all waves (therefore, if the configuration is overwritten at the source level it will be ignored).

Allocation waves times can be configured either at a root level or at an aggregate level. When defined at root level this implies that all existing sources associated to the aggregate will be considered as available during each one of the waves.

In order to disable a specific source for a specific time, the configuration key will have to be removed under the specific source node as follows:

  1. In the Scope dropdown select the source
  2. Navigate to the configuration: Sourcing > Active Wave Times
  3. Uncheck the input field DEFAULT
  4. Remove the wave by clicking trash icon

Remove Wave

If root does not have any configured wave, but there are configured waves at the aggregate level, this means that none of the sources automatically inherit the setting, therefore you can assume that is disabled. In this case, each one of the sources has to define the exact same waves that exist for the aggregate level. In order to optimize the creation of configurations, they can be defined in the MetaSource, so it is inherited by all children sources.

Force sourcing decision at order line

It is also possible to define the source to be used at the order line level, this forces the sourcing. If the source_location is defined then other available options for sourcing are ignored, and the lines are always forced to the defined source.

In case the stock is not available in the source_location the line is backordered until stock becomes available in this source.

Key Value Message Description
Force Source Location (optional) magento.sales.order_management.create With the order creation is possible to pre-define the source location that will be shipping the items. Can be used for both home delivery or ISPU orders (this last only in case ship to store feature is enabled)

Integration order flow

The following are examples for an order flow illustrating the four steps previously explained.

Alt Text

Alt Text