distinct

This page explains how to use the distinct operator function in APL.

The distinct operator in APL (Axiom Processing Language) returns a unique set of values from a specified field or set of fields. This operator is useful when you need to filter out duplicate entries and focus only on distinct values, such as unique user IDs, event types, or error codes within your datasets. Use the distinct operator in scenarios where eliminating duplicates helps you gain clearer insights from your data, like when analyzing logs, monitoring system traces, or reviewing security incidents.

For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

Splunk SPL users

In Splunk’s SPL, the dedup command is often used to retrieve distinct values. In APL, the equivalent is the distinct operator, which behaves similarly by returning unique values but without necessarily ordering them.

Splunk example

index=web_logs
| dedup user_id

ANSI SQL users

In ANSI SQL, you use SELECT DISTINCT to return unique rows from a table. In APL, the distinct operator serves a similar function but is placed after the table reference rather than in the SELECT clause.

SQL example

SELECT DISTINCT user_id FROM web_logs;

Usage

Syntax

| distinct FieldName1 [, FieldName2, ...]

Parameters

• FieldName1, FieldName2, ...: The fields to include in the distinct operation. If you specify multiple fields, the result will include rows where the combination of values across these fields is unique.

Returns

The distinct operator returns a dataset with unique values from the specified fields, removing any duplicate entries.

Use case examples

Log analysis

In this use case, the distinct operator helps identify unique users who made HTTP requests in a system.

Query

['sample-http-logs']
| distinct id

Run in Playground

Output

id
user_123
user_456
user_789

This query returns a list of unique user IDs that have made HTTP requests, filtering out duplicate user activity.

In this use case, the distinct operator helps identify unique users who made HTTP requests in a system.

Query

['sample-http-logs']
| distinct id

Run in Playground

Output

id
user_123
user_456
user_789

This query returns a list of unique user IDs that have made HTTP requests, filtering out duplicate user activity.

Here, the distinct operator is used to identify all unique services involved in traces.

Query

['otel-demo-traces']
| distinct ['service.name']

Run in Playground

Output

service.name
frontend
checkoutservice
productcatalogservice

This query returns a distinct list of services involved in traces.

In this example, you use the distinct operator to find unique HTTP status codes from security logs.

Query

['sample-http-logs']
| distinct status

Run in Playground

Output

status
200
404
500

This query provides a distinct list of HTTP status codes that occurred in the logs.

List of related operators

• count: Returns the total number of rows. Use it to count occurrences of data rather than filtering for distinct values.
• summarize: Allows you to aggregate data and perform calculations like sums or averages while grouping by distinct values.
• project: Selects specific fields from the dataset. Use it when you want to control which fields are returned before applying distinct.