Configuring task concurrency

Data Operator executes certain operations in parallel to reduce processing time.

By default, up to three concurrent tasks are executed.

The number of concurrent tasks can be configured by setting the environment variable DATAOP_CONCURRENT_TASKS_LIMIT.

Adjusting this value allows you to balance performance and memory usage:

  • Lower values reduce memory consumption and help avoid out-of-memory errors.

  • Higher values can improve processing speed on systems with sufficient CPU performance and available RAM.

If the environment variable is not set, Data Operator uses the default limit of three concurrent tasks.

Setting the environment variable with Command Prompt (cmd)

  1. Open the Start menu.

  2. Type cmd and press Enter.

  3. In the Command Prompt window, enter the following command:

    setx DATAOP_CONCURRENT_TASKS_LIMIT <value>

    Replace <value> with the desired number of concurrent tasks.

  4. Press Enter to execute the command.

  5. Close the Command Prompt window.

  6. Restart any open Command Prompt or Data Operator sessions so that the new environment variable is applied.

EXAMPLE

To limit processing to a single concurrent task:

setx DATAOP_CONCURRENT_TASKS_LIMIT 1

To verify the setting in a new Command Prompt, execute: echo %DATAOP_CONCURRENT_TASKS_LIMIT%