Recommendations AI data ingestionRecommendations AI data ingestionTechnical Solutions Consultant

d. BigQuery import

Just as you can import products from BQ in the merchant center schema, you can also create a BigQuery table using the Retail product schema. The product schema definition for BigQuery is available here. The Merchant Center Big Query schema can be used whether or not you transfer the data from Merchant Center, but it is not the full schema for retail. It doesn’t include custom attributes for example. So using the Retail Schema allows you to import all possible fields.

Importing from BigQuery is useful if your product catalog is already in BigQuery. You can also create a view that matches the Retail schema, and import from the view, pulling data from existing tables as necessary.

For Merchant Center, Cloud Storage and BigQuery imports, the import itself can be triggered through the Admin Console UI, or via the import API call. When using the API, the schema needs to be specified with the dataSchema attribute as product or product_merchant_center accordingly.

e. API import & product management

You can also import and modify catalog items directly via API. This is useful to make changes to products in realtime for example, or if you want to integrate with an existing catalog management system. 

The inline import method is very similar to GCS import: you simply construct a list of products in the Retail Schema format, and call the products.import method API to submit the products. Like with GCS, existing products are overwritten and new products are created. Currently the import method can import up to 100 products per call.

There is also the option to manage products individually with the API, using get, create, patch, and delete methods.

All of the API calls can be done using HTTP/REST or gRPC, but using the retail client libraries for the language of your choice may be the easiest option. The documentation currently has many examples using curl with the REST API, but the client libraries are usually preferred for production use.

2. Live user events

Once your catalog is imported you’ll need to start sending user events to the Retail API. Since recommendations are personalized in real-time based on recent activity, user events should be sent in real-time as they are occurring. Typically, you’ll want to start sending live, real-time events and then optionally backfill historical events before training any  models.

There are currently 4 event types used by the Recommendations AI models:

Not all models require all of these events, but it is recommended to send all of these if possible.

Note the “minimum required” fields for each event. As with the product schema, the user event schema also has many fields, but only a few are required. A typical event might looks like this:

Leave a Comment