Introduction

In Adobe Commerce, the seamless exchange of data between your e-commerce store is essential for maintaining a dynamic and efficient online presence. Our comprehensive synchronization and data loading capabilities empower you to connect your Akeneo PIM with your store, ensuring that your product information is consistently up-to-date. This section provides you with a deep dive into the intricacies of synchronization and data loading, enabling you to harness the full potential of Adobe Commerce's robust connector functionalities.

Connector Performance and scalability

Understanding Connector Performance

Customers frequently inquire about the metrics and performance of Akeneo Connectors. Unfortunately, gauging performance is a multifaceted endeavor. In general, it's important to note that three primary criteria significantly influence the performance of the connector:

1. Catalog Volume: While catalog size is an obvious consideration, it's not the sole factor at play.

2. Catalog Complexity: Importing 500,000 products can be straightforward when the product data is minimal (comprising just a name, description, and image). However, if the products are intricate, featuring numerous attributes, reference entities, variations, and the like, importing even 2,000 products can become a laborious and time-consuming process.

3. Adobe Commerce Limitations: Additionally, Adobe Commerce has its own constraints, as previously discussed in the introduction.

Optimizing Connector Performance

To enhance your connector's performance, we recommend the following strategies:

1. Direct Data Mapping: Always prioritize direct data mapping between Akeneo PIM and Adobe Commerce. This approach eliminates the need to rely on Adobe Commerce for data conversion, thereby boosting system efficiency.

2. Asset Management: Consider leveraging a Digital Asset Management (DAM) system and the Asset Manager (utilizing media links and file links) instead of uploading image binaries directly into the PIM. By saving URLs rather than high-definition images, you can significantly reduce processing time.

3. Data Filtering: Scrutinize the amount of data to be transferred to Adobe Commerce. You may have attributes in Akeneo PIM that are unnecessary for your storefront. Eliminating irrelevant data during synchronization can expedite the import process. Check the synchronization scope paragraph below for more information. 

Basic Technical information on Akeneo connector data transfer

• Data Transfer: Data transfer is facilitated through the use of an API to XML format, ensuring seamless communication between systems. Please refer to the synchronizing data section for more information. 
• Temporary tables transfer:
  • Akeneo Connector for Adobe Commerce inserts all PIM data into a temporary table. Then, data is manipulated (mapping,...) within this temporary table in SQL. Finally, the modified content is directly inserted into SQL in Adobe Commerce tables.
  • Even if raw SQL insertion is not the way you usually import data into a system, it is still faster than anything else, especially with the volume of data within a full Akeneo catalog. This results in a significant time saving for your imports.
  • Table Engine MyISAM: In the development of the connector, the choice was made to utilize the MyISAM Engine, as opposed to InnoDB, for improved performance in data reading and writing. With MyISAM, the connector can store Akeneo data in temporary tables, limited to 1250 columns. In contrast, InnoDB restricts this to only 160 columns. Each column is dedicated to a scopable or localizable attribute. For example, an attribute available in three languages would have three corresponding columns in the table.

Akeneo connector synchronization feature set

Synchronization panel 

Each job is responsible for importing PIM architecture and data into Adobe Commerce. The jobs order above should be observed to build your catalog properly. You could skip steps, but please be careful if you do. For example, if you want to import attribute options and you have created new attributes in your PIM, you will first need to import attributes or it will result in an error. Please check your data before importing it!

Akeneo Connector for Adobe Commerce is composed by 5 main jobs (5 main processes):

1. Category
2. Family
3. Attribute
4. Option
5. Product

Each job can be triggered manually or automatically on its own.You will find below the technical details for each process:

Synchronization scope

Synchronization scope for our connector defines the extent of data exchange and coordination between your Akeneo PIM and your Adobe Commerce. It specifies what data are included in the synchronization process. By customizing the synchronization scope, you have the flexibility to tailor your integration to match your business needs precisely.

Full or differential / Delta

When importing product data, you have two options to choose from:

1. Full Import: This is the most time-consuming import process because it synchronizes everything, including media, between Akeneo PIM and Adobe Commerce. However, the full import typically occurs mainly once during the initial synchronization for building your Adobe Commerce catalog. Subsequently, it is often automated for nightly resynchronization. The three performance criteria mentioned earlier significantly influence this process.

2. Partial/Differential/Delta Import: The system can synchronize only new products when necessary. However, for certain data reconstruction on the Adobe Commerce side, some tasks must be run as a full import (e.g., fully importing categories even in a differential import). Nevertheless, the differential import process reduces the required import time and is employed daily to resynchronize data that has changed within the day. This type of import is usually initiated manually.

To make an informed choice between a full import and a differential import, you should define your data update policy and set the threshold accordingly.

The choice of full or differential import within Adobe Commerce connector is to be configured within the connector configuration page and not in the jobs. To select the import mode in the connector configuration, whenever applicable, here are the steps to reproduce:

• Category: only full import
• Family: go to the filter family section
• Attribute and options: go to the attribute and attributes options section
• Products: go to the product and product model filtering section

Apply filters to your imports

In order to select and narrow the data you want to import, you can filter the entities within the connector configuration. Go to the setup and use the connector section for more details on filtering possibilities.

Synchronization type

Manually trigger jobs 

The connector allows you to launch your synchronizations manually. To launch a synchronization manually, follow the steps below:

Since 102.0.0

Jobs are now run asynchronously and will be scheduled before being triggered.

To ensure that jobs are run as fast as possible, a new crontask ("akeneo_connector_launch_scheduled_job") has been added to the connector, and will check every minute if a job is ready to be triggered.

When a job is manually scheduled, the job status is changed to "Scheduled", and the Akeneo Connector crontask will trigger it in background under a minute.

In order to manually run the jobs, you can follow these steps:

1. Make sure that Adobe Commerce CRON are correctly setup and running (CRON Documentation)
2. Click on the "Schedule Job" link to schedule one job or select all the jobs you want to schedule and use the action "Schedule Job"
3. Wait for the Akeneo Connector Cron task to run the job(s) in background

When a job is currently running, its status will be switched to "Processing".

If a job execution fails, its status will be switched to "Error", otherwise it will be set to "Success".

When manually scheduling multiple jobs together, the crontask will manage the correct order of the jobs, and trigger them accordingly:

1. Category
2. Family
3. Attribute
4. Option
5. Product

You can only schedule a job if its status is: "Pending", "Success" or "Error"

 

A new Adobe Commerce cron group has been created for the connector cron tasks.

You can find the configurations under "Stores > Configuration > Advanced > System > Cron configuration options for group:akeneo_connector"

 

You can reset a job status by using the grid action: "Reset Status". The job status will be set to "Pending".

This action can be used if an independant bug or incident happens during a job execution and that the job is stuck in "Processing" status.

 

 

 

Before 102.0.0

You can follow this procedure:

1. Select the job you want to run (import type),
2. Then click on the Import button to trigger the selected job,
3. Wait until the process ends.

The Job import process is completed when its status changes to Import complete in the console window.

If it fails, you could see a red error in the console window. Please take a screenshot of the console window: you will probably need it to communicate with your technical team or to open a ticket for our Support team via our Helpdesk platform (Akeneo Connector for Adobe Commerce Enterprise Edition only).

 

 

 

Scheduled or asynchronous synchronization

With the connector, you enjoy the convenience of scheduling your imports to occur on a recurring basis. This feature provides you with the flexibility to automate and streamline the process of bringing data into your system at specified intervals, ensuring that your product information remains up-to-date and synchronized with external sources. Whether you need daily updates, weekly refreshes, or any other custom schedule, our connector empowers you to tailor your import routines to suit your specific business needs and operational efficiency. The scheduled synchronization need to be expressed in CRON.

There are two ways to schedule import tasks in the connector : creating a Server Crontab or, for the Enterprise Edition, using the Magento Crontab within the UI of the command line. 

Adobe Commerce Crontab

Available since 103.5.0 version of the Akeneo connector for Adobe Commerce.

You can use the native Magento Cron system within the UI of the command line. 

This feature is only available for the Akeneo connector for Adobe Commerce Enterprise Edition.

 

Requirements

The Adobe Commerce Cron system must be enabled: Configure cron jobs

Configuration

You need to add your Cron Expression to the jobs page in the Adobe Commerce admin: 

System > Akeneo Connector > Jobs

In the Cron column, set the desired Cron Expression. After the update, the Schedule At value will be automatically calculated from the Cron Expression and will be refreshed after each import.

The task will be always launched except if the job has the “Processing” status. Indeed, the job is already running in that case so the import can not be launched again as it is already processing.

 

Cron Expression

If you need help writing a Cron expression, you can also use the “Cron Guru” online tool: crontab.guru

 

 

bin/magento akeneo_connector:import --code=category
bin/magento akeneo_connector:import --code=family
bin/magento akeneo_connector:import --code=attribute
bin/magento akeneo_connector:import --code=option
bin/magento akeneo_connector:import --code=product

crontab -e

0 1 * * * /var/www/magento/bin/magento akeneo_connector:import --code=product

0 22 * * * /usr/bin/php /path/to/magento/current/bin/magento akeneo_connector:import --code=import-type >> /path/to/magento/current/var/log/akeneo_connector_import_type.cron.log`
 

php bin/magento akeneo_connector:import --code=import-type
 

php bin/magento akeneo_connector:import --code=import-type1,import-type2,import-type3
 

Synchronization status

The following synchronization statuses are available in the Adobe Commerce connector.

Starting from the version 103.3.0, The job grid will be automatically reloaded when a job is "Scheduled" or "Processing" in order to update in real time the job status. The automatic reload can be disabled from the connector configuration page, in Advanced > Enable Job grid auto reload.

You can reset a job status by using the grid action: "Reset Status". The job status will be set to "Pending".

This action can be used if an independant bug or incident happens during a job execution and that the job is stuck in "Processing" status.

Product deletion

Deleting synchronized products in Adobe Commerce is not handled by the connector. If a product is deleted on the Akeneo side, it must be manually deleted in Adobe Commerce. 
If a product is deleted in Adobe Commerce, it must be manually deleted in Akeneo. Otherwise, it will be recreated by the app during the next synchronization.

The recommended steps are:

1. set the product status to disable (not enabled)
2. synchronize your data with the command line of the connector. Product are deactivated - not enabled anymore. 
3. if needed, delete the product in the PIM and/Or delete the products in Adobe Commerce.

Empty attributes

You can reset the value of an attribute in Adobe Commerce after synchronization from Akeneo PIM.

If you clear the content of a text attribute (make it empty) in Akeneo PIM it will be synchronized by Akeneo Connector for Adobe Commerce.

For simple select attribute type, you need to have a “none” option for this attribute to be able to reset it (because you can’t « unselect » a simple select). It will then be synchronized by Akeneo Connector for Adobe Commerce.

For multi-select, you can clean all option content in Akeneo PIM, it will be synchronized by Akeneo Connector for Adobe Commerce.

Where to find import jobs in Adobe Commerce?

Since 102.0.0

You can find the jobs under:

1. Adobe Commerce SYSTEM menu
2. Then click on Jobs in Akeneo Connector:

Statuses displays as below