Send OpenTelemetry data from a Django app to Axiom

This guide explains how to send OpenTelemetry data from a Django app to Axiom using the Python OpenTelemetry SDK.

Prerequisites

• Create an Axiom account.
• Create a dataset in Axiom where you send your data.
• Create an API token in Axiom with permissions to update the dataset you have created.

• Install Python version 3.7 or higher.

Install required dependencies

Install the necessary Python dependencies by running the following command in your terminal:

pip install django opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-django

Alternatively, you can add these dependencies to your requirements.txt file:

django
opentelemetry-api
opentelemetry-sdk
opentelemetry-exporter-otlp-proto-http
opentelemetry-instrumentation-django

Then, install them using the command:

pip install -r requirements.txt

Get started with a Django project

1. Create a new Django project if you don’t have one already:

django-admin startproject your_project_name

2. Go to your project directory:

cd your_project_name

3. Create a Django app:

python manage.py startapp your_app_name

Set up OpenTelemetry Tracing

Update manage.py to initialize tracing

This code initializes OpenTelemetry instrumentation for Django when the project is run. Adding DjangoInstrumentor().instrument() ensures that all incoming HTTP requests are automatically traced, which helps in monitoring the app’s performance and behavior without manually adding trace points in every view.

# manage.py

#!/usr/bin/env python
import os
import sys
from opentelemetry.instrumentation.django import DjangoInstrumentor

def main():
    """Run administrative tasks."""
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project_name.settings')

    # Initialize OpenTelemetry instrumentation
    DjangoInstrumentor().instrument()

    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)

if __name__ == '__main__':
    main()

Create exporter.py for tracer configuration

This file configures the OpenTelemetry tracing provider and exporter. By setting up a TracerProvider and configuring the OTLPSpanExporter, you define how and where the trace data is sent. The BatchSpanProcessor is used to batch and send trace spans efficiently. The tracer created at the end is used throughout the app to create new spans.

# exporter.py

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

# Define the service name resource
resource = Resource(attributes={
    SERVICE_NAME: "your-service-name"  # Replace with your actual service name
})

# Create a TracerProvider with the defined resource
provider = TracerProvider(resource=resource)

# Configure the OTLP/HTTP Span Exporter with necessary headers and endpoint
otlp_exporter = OTLPSpanExporter(
    endpoint="https://AXIOM_DOMAIN/v1/traces",
    headers={
        "Authorization": "Bearer API_TOKEN",  # Replace with your actual API token
        "X-Axiom-Dataset": "DATASET_NAME"    # Replace with your dataset name
    }
)

# Create a BatchSpanProcessor with the OTLP exporter
processor = BatchSpanProcessor(otlp_exporter)
provider.add_span_processor(processor)

# Set the TracerProvider as the global tracer provider
trace.set_tracer_provider(provider)

# Define a tracer for external use
tracer = trace.get_tracer("your-service-name")

Replace AXIOM_DOMAIN with api.axiom.co if your organization uses the US region, and with api.eu.axiom.co if your organization uses the EU region. For more information, see Regions.

Replace API_TOKEN with the Axiom API token you have generated. For added security, store the API token in an environment variable.

Replace DATASET_NAME with the name of the Axiom dataset where you want to send data.

Use the tracer in your views

In this step, modify the Django views to use the tracer defined in exporter.py. By wrapping the view logic within tracer.start_as_current_span, you create spans that capture the execution of these views. This provides detailed insights into the performance of individual request handlers, helping to identify slow operations or errors.

# views.py

from django.http import HttpResponse
from .exporter import tracer  # Import the tracer

def roll_dice(request):
    with tracer.start_as_current_span("roll_dice_span"):
        # Your logic here
        return HttpResponse("Dice rolled!")

def home(request):
    with tracer.start_as_current_span("home_span"):
        return HttpResponse("Welcome to the homepage!")

Update settings.py for OpenTelemetry instrumentation

In your Django project’s settings.py, add the OpenTelemetry Django instrumentation. This setup automatically creates spans for HTTP requests handled by Django:

# settings.py

from pathlib import Path
from opentelemetry.instrumentation.django import DjangoInstrumentor

DjangoInstrumentor().instrument()

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent

Update the app’s urls.py to include the views

Include your views in the URL routing by updating urls.py Updating urls.py with these entries sets up the URL routing for the Django app. It connects the URL paths to the corresponding view functions. This ensures that when users visit the specified paths, the corresponding views are executed, and their spans are created and sent to Axiom for monitoring.

# urls.py

from django.urls import path
from .views import roll_dice, home

urlpatterns = [
    path('', home, name='home'),
    path('rolldice/', roll_dice, name='roll_dice'),
]

Run the project

Run the command to start the Django project:

python3 manage.py runserver

In your browser, go to http://127.0.0.1:8000/rolldice to interact with your Django app. Each time you load the page, the app displays a message and sends the collected traces to Axiom.

Send data from an existing Django project

Manual instrumentation

Manual instrumentation in Python with OpenTelemetry involves adding code to create and manage spans around the blocks of code you want to trace. This approach allows for precise control over the trace data.

1. Install necessary OpenTelemetry packages to enable manual tracing capabilities in your Django app.

   pip install django opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-django
   

2. Set up OpenTelemetry in your Django project to manually trace app activities.

   # otel_config.py
   from opentelemetry import trace
   from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
   from opentelemetry.sdk.resources import Resource
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import BatchSpanProcessor
   
   def configure_opentelemetry():
       resource = Resource(attributes={"service.name": "your-django-app"})
       trace.set_tracer_provider(TracerProvider(resource=resource))
       otlp_exporter = OTLPSpanExporter(
           endpoint="https://AXIOM_DOMAIN/v1/traces",
           headers={"Authorization": "Bearer API_TOKEN", "X-Axiom-Dataset": "DATASET_NAME"}
       )
       span_processor = BatchSpanProcessor(otlp_exporter)
       trace.get_tracer_provider().add_span_processor(span_processor)
       return trace.get_tracer(__name__)
   
   tracer = configure_opentelemetry()
   

   Replace AXIOM_DOMAIN with api.axiom.co if your organization uses the US region, and with api.eu.axiom.co if your organization uses the EU region. For more information, see Regions.

   Replace API_TOKEN with the Axiom API token you have generated. For added security, store the API token in an environment variable.

   Replace DATASET_NAME with the name of the Axiom dataset where you want to send data.

3. Configure OpenTelemetry to your Django settings to capture telemetry data upon app startup.

# settings.py
from otel_config import configure_opentelemetry
configure_opentelemetry()

4. Manually instrument views to create custom spans that trace specific operations within your Django app.

# views.py
from django.http import HttpResponse
from otel_config import tracer

def home_view(request):
    with tracer.start_as_current_span("home_view") as span:
        span.set_attribute("http.method", request.method)
        span.set_attribute("http.url", request.build_absolute_uri())
        response = HttpResponse("Welcome to the home page!")
        span.set_attribute("http.status_code", response.status_code)
        return response

5. Apply manual tracing to database operations by wrapping database cursor executions with OpenTelemetry spans.

# db_tracing.py
from django.db import connections
from otel_config import tracer

class TracingCursorWrapper:
    def __init__(self, cursor):
        self.cursor = cursor
    def execute(self, sql, params=None):
        with tracer.start_as_current_span("database_query") as span:
            span.set_attribute("db.statement", sql)
            span.set_attribute("db.type", "sql")
            return self.cursor.execute(sql, params)
    def __getattr__(self, attr):
        return getattr(self.cursor, attr)

def patch_database():
    for connection in connections.all():
        connection.cursor_wrapper = TracingCursorWrapper

# settings.py
from db_tracing import patch_database
patch_database()

Automatic instrumentation

Automatic instrumentation in Django with OpenTelemetry simplifies the process of adding telemetry data to your app. It uses pre-built libraries that automatically instrument the frameworks and libraries.

1. Install required packages that support automatic instrumentation.

   pip install django opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-django
   

2. Automatically configure OpenTelemetry to trace Django app operations without manual span management.

   # otel_config.py
   from opentelemetry import trace
   from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
   from opentelemetry.instrumentation.django import DjangoInstrumentor
   from opentelemetry.sdk.resources import Resource
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import BatchSpanProcessor
   
   def configure_opentelemetry():
       resource = Resource(attributes={"service.name": "your-django-app"})
       trace.set_tracer_provider(TracerProvider(resource=resource))
       otlp_exporter = OTLPSpanExporter(
           endpoint="https://AXIOM_DOMAIN/v1/traces",
           headers={"Authorization": "Bearer API_TOKEN", "X-Axiom-Dataset": "DATASET_NAME"}
       )
       span_processor = BatchSpanProcessor(otlp_exporter)
       trace.get_tracer_provider().add_span_processor(span_processor)
       DjangoInstrumentor().instrument()
   

   Replace AXIOM_DOMAIN with api.axiom.co if your organization uses the US region, and with api.eu.axiom.co if your organization uses the EU region. For more information, see Regions.

   Replace API_TOKEN with the Axiom API token you have generated. For added security, store the API token in an environment variable.

   Replace DATASET_NAME with the name of the Axiom dataset where you want to send data.

3. Initialize OpenTelemetry in Django to capture telemetry data from all HTTP requests automatically.

# settings.py
from otel_config import configure_opentelemetry
configure_opentelemetry()

4. Update manage.py to include OpenTelemetry initialization, ensuring that tracing is active before the Django app fully starts.

#!/usr/bin/env python
import os
import sys
def main():
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings')
    from otel_config import configure_opentelemetry
    configure_opentelemetry()
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError("Couldn't import Django.") from exc
    execute_from_command_line(sys.argv)
if __name__ == '__main__':
    main()

5. (Optional) Combine automatic and custom manual spans in Django views to enhance trace details for specific complex operations.

# views.py
from opentelemetry import trace
tracer = trace.get_tracer(__name__)
def complex_view(request):
    with tracer.start_as_current_span("complex_operation"):
        result = perform_complex_operation()
    return HttpResponse(result)

Reference

List of OpenTelemetry trace fields

Field Category	Field Name	Description
General Trace Information
	_rowId	Unique identifier for each row in the trace data.
	_sysTime	System timestamp when the trace data was recorded.
	_time	Timestamp when the actual event being traced occurred.
	trace_id	Unique identifier for the entire trace.
	span_id	Unique identifier for the span within the trace.
	parent_span_id	Unique identifier for the parent span within the trace.
HTTP Attributes
	attributes.http.method	HTTP method used for the request.
	attributes.http.status_code	HTTP status code returned in response.
	attributes.http.route	Route accessed during the HTTP request.
	attributes.http.scheme	Protocol scheme (HTTP/HTTPS).
	attributes.http.url	Full URL accessed during the HTTP request.
User Agent
	attributes.http.user_agent	User agent string, providing client software and OS.
Custom Attributes
	attributes.custom[“http.host”]	Host information where the HTTP request was sent.
	attributes.custom[“http.server_name”]	Server name for the HTTP request.
	attributes.custom[“net.peer.ip”]	IP address of the peer in the network interaction.
Network Attributes
	attributes.net.host.port	Port number on the host receiving the request.
Operational Details
	duration	Time taken for the operation, typically in microseconds or milliseconds.
	kind	Type of span (For example, server, internal).
	name	Name of the span, often a high-level title for the operation.
Scope and Instrumentation
	scope	Instrumentation scope, (For example., opentelemetry.instrumentation.django.)
Service Attributes
	service.name	Name of the service generating the trace, typically set as the app or service name.
Telemetry SDK Attributes
	telemetry.sdk.language	Programming language of the SDK used for telemetry, typically ‘python’ for Django.
	telemetry.sdk.name	Name of the telemetry SDK, for example., OpenTelemetry.
	telemetry.sdk.version	Version of the telemetry SDK used in the tracing setup.

List of imported libraries

The exporter.py file and other relevant parts of the Django OpenTelemetry setup import the following libraries:

exporter.py

This module creates and manages trace data in your app. It creates spans and tracers which track the execution flow and performance of your app.

from opentelemetry import trace

TracerProvider acts as a container for the configuration of your app’s tracing behavior. It allows you to define how spans are generated and processed, essentially serving as the central point for managing trace creation and propagation in your app.

from opentelemetry.sdk.trace import TracerProvider

BatchSpanProcessor is responsible for batching spans before they’re exported. This is an important aspect of efficient trace data management as it aggregates multiple spans into fewer network requests, reducing the overhead on your app’s performance and the tracing backend.

from opentelemetry.sdk.trace.export import BatchSpanProcessor

The Resource class is used to describe your app’s service attributes, such as its name, version, and environment. This contextual information is attached to the traces and helps in identifying and categorizing trace data, making it easier to filter and analyze in your monitoring setup.

from opentelemetry.sdk.resources import Resource, SERVICE_NAME

The OTLPSpanExporter is responsible for sending your app’s trace data to a backend that supports the OTLP such as Axiom. It formats the trace data according to the OTLP standards and transmits it over HTTP, ensuring compatibility and standardization in how telemetry data is sent across different systems and services.

from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

manage.py

The DjangoInstrumentor module is used to automatically instrument Django applications. It integrates OpenTelemetry with Django, enabling automatic creation of spans for incoming HTTP requests handled by Django, and simplifying the process of adding telemetry to your app.

from opentelemetry.instrumentation.django import DjangoInstrumentor

views.py

This import brings in the tracer instance defined in exporter.py, which is used to create spans for tracing the execution of Django views. By wrapping view logic within tracer.start_as_current_span, it captures detailed insights into the performance of individual request handlers.

from .exporter import tracer