Hosting Variables

While the API offers ways to configure Generator’s behavior as an application, most of the aspects that are relevant to a hosting or infrastructure team are controlled with environment variables. These variables are mostly easily defined in a .env file, but you can also pass them directly to your Docker container or runtime environment as needed.

Example File

Below is an example .env file to help you get started. Variables with comments but no default values require your input before launching the application; the others are provided with reasonable defaults. Be sure to review and customize these settings to suit your environment.

HOSTING__PROJECT_NAME=Rustici Generator

# A JSON formatted array of domains that CORS requests are accepted from
# This value will block all CORS access
HOSTING__BACKEND_CORS_ORIGINS=[]

# The domain that will be used to access your Generator instance.
# This is critical for Engine integrations - Engine uses this value to send webhook notifications
# to Generator. If this value is incorrect or missing, Engine webhooks will fail to reach Generator.
# The value should be the domain of your Generator instance (e.g., `generator.example.com`) with no protocol or trailing slash,
# and the domain must be accessible from the Engine instance.
HOSTING__DOMAIN=

# The /metrics endpoint can be disabled if desired.
# FYI, the /metrics endpoint only responds when accessed directly
# (i.e. NOT through a proxy or load balancer, as determined by the presence of any "x-forwarded-*" request headers)
HOSTING__ENABLE_METRICS=True

# Sets a maximum to the count of fields that can be generated at once.
# Can be valuable to ensure that client's don't accidentally request an absurdly large number of fields.
HOSTING__GENERATION_COUNT_MAX=5

# An API credential that is created during start up.
# You will use this to access the API the first time.
# Set these to something secure.
# After the application has been started once, these env vars are still required to start the app,
# but you can change/delete the var values to avoid storing a copy of your credentials on disk (unless SECURITY__DISABLE_API_AUTH is True).
SECURITY__FIRST_CREDENTIAL=
SECURITY__FIRST_CREDENTIAL_PASSWORD=

# A base64-encoded, 32-character key used to encrypt sensitive information stored in the app database
# Set to something random and secure (example generation command: "head -c 24 /dev/urandom | base64")
# If you ever want to rotate your encryption keys, see the SECURITY__PREVIOUS_ENCRYPTION_KEYS setting below
SECURITY__ENCRYPTION_KEY=

# After rotating your encryption keys, Generator still needs to be able to decode records that were encrypted with your previous key.
# After changing the value of SECURITY__ENCRYPTION_KEY, adding the previous value to this JSON formatted array
# will allow that.
SECURITY__PREVIOUS_ENCRYPTION_KEYS=[]

# A secret key used to sign the API's auth tokens
# Set to something random and secure
SECURITY__SECRET_KEY=

# API key required to access the /api/v1/health/check_up endpoint.
# This endpoint provides detailed health information and should be protected.
# Set to a random, secure value. Requests must include this value in the X-API-Key header.
# Example: curl -H "X-API-Key: your-api-key-here" http://localhost/api/v1/health/check_up
SECURITY__HEALTH_CHECK_API_KEY=

# How long in minutes should API auth tokens live before expiring
SECURITY__ACCESS_TOKEN_EXPIRE_MINUTES=60

# Options are DEBUG, INFO, WARN, ERROR, and FATAL
LOGGING__DEFAULT_LEVEL=INFO

# Configures the logging format for the application.
#   Supports various formatting options, see docs for details: https://loguru.readthedocs.io/en/stable/api/logger.html#record
#   Defaults to "{time:YYYY-MM-DDTHH:mm:ss.SSS} {level} {name}:{function}:{line} {message}" on the backend service,
#   and the worker service adds the "{process}" and "{thread}" columns to help trace logs from one thread in a busy log file
# LOGGING__FORMAT="{time:YYYY-MM-DDTHH:mm:ss.SSS} {level} {name}:{function}:{line} {message}"

# Details for connecting to the app database
DB__HOST=
DB__PORT=5432
DB__USER=
DB__PASSWORD=
DB__DATABASE=generator
DB__ENGINE=postgresql
DB__DRIVER=psycopg2

# Details of your Rabbit MQ broker
QUEUE__USER=
QUEUE__HOST=
QUEUE__PASSWORD=
QUEUE__PORT=5672
# Use the following settings if you want to configure a TLS connection for your MQ broker
# QUEUE__PROTOCOL=amqps
# QUEUE__PORT=5671
# If necessary, you can configure Generator with the SSL cert of your queue broker.
# You'll need to mount the certificate onto the containers, then configure this setting
# with the path of that certificate within the containers:
# QUEUE__CERT_FILE=

# The name of the S3 bucket where Generator should store its content
AWS_S3_BUCKET=

# Controls how many threads are used to execute tasks from the queue
# We recommend a starting point of roughly 1 thread per GB of memory on your worker node(s).
# However, requirements may vary due to your usage, so this can be increased if your host's metrics
# aren't being stressed.
# In particular, PDF parsing uses a lot of memory, so if you aren't doing much of that, you probably have room
# to increase the number of threads.
THREADS__DRAMATIQ_THREADS=2

# Controls how many Dramatiq process run in a worker node.
# Each process will contain $THREADS__DRAMATIQ_THREADS number of threads,
# so this setting can drastically increase resource requirements
THREADS__DRAMATIQ_PROCESSES=1

Optional Settings

Here are some optional settings that you can change if necessary. Specifying them is not necessary in all cases, as they all have reasonable defaults.

# Set these if you're not using something like IAM roles/EC2 instance profiles
AWS_ACCESS_KEY_ID=
AWS_REGION=
AWS_SECRET_ACCESS_KEY=
AWS_SESSION_TOKEN=

# Set this if you want to override the default endpoint. For example, sending the Bedrock traffic through your own LLM service
# Example value: https://bedrock-runtime.us-east-1.amazonaws.com
AWS_OVERRIDE_BEDROCK_ENDPOINT_URL

# The minimum size of the connection pool
DB__POOL_SIZE=5
# The maximum size of the connection pool
DB__MAX_OVERFLOW=10
# How long to wait for a connection from the pool before timing out
DB__POOL_TIMEOUT=30

# The ID and version of an AWS Bedrock Guardrail, if you'd like to use one
GENERATION__BEDROCK_GUARDRAIL_ID=
GENERATION__BEDROCK_GUARDRAIL_VERSION=

# If you want to entirely disable all uses of LLMs and machine learning, this setting can be disabled.
# Only useful if you are exlusively interested in the text extraction parts of Generator.
# Please note:
# - This can have a negative affect on extracted text quality, as the LLM service is used to improve
#   audio/video transcription quality
# - This will functionally disable semantic search, as we rely on AWS Bedrock's Cohere service
#   to create semantic embeddings of the content during import and search
LLM__USAGE_ENABLED=True

# The maximum number of tokens to allow in response to prompts for the Claude model
# Useful for enforcing a cap on model output to restrict generation costs,
# but can be extended if you're running into errors about responses running into this limit.
# LLM__CLAUDE_MAX_TOKENS=4000

# How long should the config cache last before being refreshed
HOSTING__CONFIG_CACHE_EXPIRY_MINUTES=5

# (disabled by default, mostly intended for use in managed hosting instances of Generator)
# When this setting is configured, Generator will copy all imported files
# to the application's S3 bucket and tag them with the configured tag.
# In S3, we recommend setting up a lifecycle rule that deletes all files with this tag
# after a certain time period.
# This can be useful for troubleshooting errors during the import process,
# especially if hosting and using the application are performed by separate teams.
HOSTING__CONTENT_IMPORT_S3_CACHE_TAG=

# The maximum number of locations to scrape during a web import
# Protects against accidentally importing way too many locations or spamming the source with requests
# Lower this for more protection, raise it for more freedom
HOSTING__WEB_IMPORT_MAX_LOCATIONS=1000

# Should the text of lessons be logged? True or false
LOGGING__LESSON_TEXT=False

# Set this if you want logs to be written to a file rather than sent to stdout
LOGGING__LOCATION=

# Can be used to customize the log level of different modules.
# i.e. LOGGING__MODULE_LEVELS='{"app.ai.ai_client":"INFO"}
LOGGING__MODULE_LEVELS=

# How many times should we retry messages from the queue after encountering a retryable error
QUEUE__MAX_RETRIES=2

# Time window tolerance in seconds for timestamp nonce validation
# Timestamps must be within this many seconds of the server's current time
# Default: 60 seconds
SECURITY__NONCE_TIME_TOLERANCE_SECONDS=60

# Whether timestamp_nonce is required for authentication requests
# Set to 'true' to require all login requests to include a timestamp_nonce parameter
# Default: false (nonce is optional but recommended)
SECURITY__NONCE_REQUIRED=false

# Disables any authorization requirements for the API, such that no authorization header is needed.
# A convienent option for deployments within secure internal networks.
# The SECURITY__FIRST_CREDENTIAL var must be referencing an active root credential if this setting is True.
# If this value is changed, the container must be restarted for it to take affect.
SECURITY__DISABLE_API_AUTH=False

# Controls the required complexity of credential passwords.
# Enforced when passwords are changed with the PUT and PATCH /credential endpoints
SECURITY__PASSWORD_REQUIRE_NUMBER=True
SECURITY__PASSWORD_REQUIRE_SPECIAL=True
SECURITY__PASSWORD_REQUIRE_LOWERCASE=True
SECURITY__PASSWORD_REQUIRE_UPPERCASE=True
SECURITY__PASSWORD_MIN_LENGTH=12

# How many failed login attempts before locking out a credential
SECURITY__LOCKOUT_FAILED_ATTEMPTS=3
# How long should a credential lockout last
SECURITY__LOCKOUT_DURATION_MINUTES=5