* Webhooks (#3607) * Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample * WIP: Start getting a Webhook page created. Just the skeleton state right now * WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now * WIP: Get library installed and make TS happy * Get a few checkboxes ready to go * Get creating and destroying working with a decent start to a frontend * Didn't mean to enable this * Remove eslint and fix other random typescript issue * Rename some events to be more realistic * Revert these changes * PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id * Allow any admin on the team to edit webhooks * Start sending some webhooks for some User events * Make sure the URL is valid * Start recording webhook deliveries * Make sure to verify if the subscription is for the type of event we are looking at * Refactor sending Webhooks and follow better webhook schema This creates a presenter to unify the format of webhooks. We also extract the sending of webhooks and recording their deliveries to a method than can be used by each of the different event type methods We also add a status to WebhookDelivery since we need to save the record before we make the HTTP request to get its id. Then once we make the request and get a response we can update the delivery with the HTTP info * Turn off a subscription that has failed for the last 25 deliveries * Get a first spec passing. Found a bug in my returning of promises so good to patch that up now * This looks nicer * Get some tests added for the processor * Add cron task to delete older webhooks * Add Document Events to the Processor * Revisions, FileOperations and Collections * Get all the server side events added to the processor and make Typescript make sure they are all accounted for * Get all the events added to the Frontend and work on styling them a bit, still needs some love though * Get UI styled up a bit * Get events wired up for webhook subscriptions * Get delete events working and test at least one variant of them * Get deletes working and actually make sure to send the model id in the webhook * Remove webhook secrets from this slice * Add disabled label for subscriptions that are disabled * Make sure to cascade the delete * Reorg this file a bit * Fix association * I removed secret for the moment * Apply Copy changes from PR Review Co-authored-by: Tom Moor <tom.moor@gmail.com> * Actually apply the copy changes TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github Co-authored-by: Tom Moor <tom.moor@gmail.com> * Update app/scenes/Settings/Webhooks.tsx Missed this copy change before Co-authored-by: Tom Moor <tom.moor@gmail.com> * Add disabled as yellow badge * Resolve frontend comments * Fixup Schema a bit and remove the dependency on the subscription * Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes * Fix WebhookDeliveries to store the responses as Text instead of blobs * Switch to text better for response bodies, this is using the helpers better and makes the code read better * Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next * Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline * Allow any team admin to see any webhook subscription for the team * Add the indexes based on our lookup patterns * Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes * Allow subscriptions to be edited after creation * Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here * I think this will get these into the translations file * Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy Co-authored-by: Tom Moor <tom.moor@gmail.com> * fix: tsc fix: Document model payload empty * fix: Revision webhook payload Add custom UA for hooks * Add webhooks icon, move under Integrations settings Some spacing fixes * Add actorId to webhook payloads * Add View and ApiKey event types * Spacing tweaks, fix team payload * fix: Webhook not disabled after 25 failures * fix: Enable webhook when editing if previously disabled * fix: Correctly store response headers * fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery * fix: Awkward payload for users.invite webhook * Add BaseEvent, ShareEvent * fix: Add share events to form * fix: Move webhook delivery cleanup to single DB call Remove some unused abstraction * Add user, collection, group context to membership webhook events Some associated refactoring Co-authored-by: Corey Alexander <coreyja@gmail.com>
526 lines
14 KiB
TypeScript
526 lines
14 KiB
TypeScript
/* eslint-disable @typescript-eslint/no-var-requires */
|
||
|
||
// Load the process environment variables
|
||
require("dotenv").config({
|
||
silent: true,
|
||
});
|
||
|
||
import {
|
||
validate,
|
||
IsNotEmpty,
|
||
IsUrl,
|
||
IsOptional,
|
||
IsByteLength,
|
||
Equals,
|
||
IsNumber,
|
||
IsIn,
|
||
IsEmail,
|
||
IsBoolean,
|
||
Contains,
|
||
MaxLength,
|
||
} from "class-validator";
|
||
import { languages } from "@shared/i18n";
|
||
import { CannotUseWithout } from "@server/utils/validators";
|
||
import Deprecated from "./models/decorators/Deprecated";
|
||
|
||
export class Environment {
|
||
private validationPromise;
|
||
|
||
constructor() {
|
||
this.validationPromise = validate(this);
|
||
}
|
||
|
||
/**
|
||
* Allows waiting on the environment to be validated.
|
||
*
|
||
* @returns A promise that resolves when the environment is validated.
|
||
*/
|
||
public validate() {
|
||
return this.validationPromise;
|
||
}
|
||
|
||
/**
|
||
* The current envionment name.
|
||
*/
|
||
@IsIn(["development", "production", "staging", "test"])
|
||
public ENVIRONMENT = process.env.NODE_ENV ?? "production";
|
||
|
||
/**
|
||
* The secret key is used for encrypting data. Do not change this value once
|
||
* set or your users will be unable to login.
|
||
*/
|
||
@IsByteLength(32, 64)
|
||
public SECRET_KEY = `${process.env.SECRET_KEY}`;
|
||
|
||
/**
|
||
* The secret that should be passed to the cron utility endpoint to enable
|
||
* triggering of scheduled tasks.
|
||
*/
|
||
@IsNotEmpty()
|
||
public UTILS_SECRET = `${process.env.UTILS_SECRET}`;
|
||
|
||
/**
|
||
* The url of the database.
|
||
*/
|
||
@IsNotEmpty()
|
||
@IsUrl({
|
||
require_tld: false,
|
||
allow_underscores: true,
|
||
protocols: ["postgres", "postgresql"],
|
||
})
|
||
public DATABASE_URL = `${process.env.DATABASE_URL}`;
|
||
|
||
/**
|
||
* The url of the database pool.
|
||
*/
|
||
@IsOptional()
|
||
@IsUrl({
|
||
require_tld: false,
|
||
allow_underscores: true,
|
||
protocols: ["postgres", "postgresql"],
|
||
})
|
||
public DATABASE_CONNECTION_POOL_URL = this.toOptionalString(
|
||
process.env.DATABASE_CONNECTION_POOL_URL
|
||
);
|
||
|
||
/**
|
||
* Database connection pool configuration.
|
||
*/
|
||
@IsNumber()
|
||
@IsOptional()
|
||
public DATABASE_CONNECTION_POOL_MIN = this.toOptionalNumber(
|
||
process.env.DATABASE_CONNECTION_POOL_MIN
|
||
);
|
||
|
||
/**
|
||
* Database connection pool configuration.
|
||
*/
|
||
@IsNumber()
|
||
@IsOptional()
|
||
public DATABASE_CONNECTION_POOL_MAX = this.toOptionalNumber(
|
||
process.env.DATABASE_CONNECTION_POOL_MAX
|
||
);
|
||
|
||
/**
|
||
* Set to "disable" to disable SSL connection to the database. This option is
|
||
* passed through to Postgres. See:
|
||
*
|
||
* https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLMODE
|
||
*/
|
||
@IsIn(["disable", "allow", "require", "prefer", "verify-ca", "verify-full"])
|
||
@IsOptional()
|
||
public PGSSLMODE = process.env.PGSSLMODE;
|
||
|
||
/**
|
||
* The url of redis. Note that redis does not have a database after the port.
|
||
* Note: More extensive validation isn't included here due to our support for
|
||
* base64-encoded configuration.
|
||
*/
|
||
@IsNotEmpty()
|
||
public REDIS_URL = process.env.REDIS_URL;
|
||
|
||
/**
|
||
* The fully qualified, external facing domain name of the server.
|
||
*/
|
||
@IsNotEmpty()
|
||
@IsUrl({ require_tld: false })
|
||
public URL = `${process.env.URL}`;
|
||
|
||
/**
|
||
* If using a Cloudfront/Cloudflare distribution or similar it can be set below.
|
||
* This will cause paths to javascript, stylesheets, and images to be updated to
|
||
* the hostname defined in CDN_URL. In your CDN configuration the origin server
|
||
* should be set to the same as URL.
|
||
*/
|
||
@IsOptional()
|
||
@IsUrl()
|
||
public CDN_URL = this.toOptionalString(process.env.CDN_URL);
|
||
|
||
/**
|
||
* The fully qualified, external facing domain name of the collaboration
|
||
* service, if different (unlikely)
|
||
*/
|
||
@IsUrl({ require_tld: false, protocols: ["http", "https", "ws", "wss"] })
|
||
@IsOptional()
|
||
public COLLABORATION_URL = this.toOptionalString(
|
||
process.env.COLLABORATION_URL
|
||
);
|
||
|
||
/**
|
||
* The port that the server will listen on, defaults to 3000.
|
||
*/
|
||
@IsNumber()
|
||
@IsOptional()
|
||
public PORT = this.toOptionalNumber(process.env.PORT);
|
||
|
||
/**
|
||
* Optional extra debugging. Comma separated
|
||
*/
|
||
public DEBUG = `${process.env.DEBUG}`;
|
||
|
||
/**
|
||
* How many processes should be spawned. As a reasonable rule divide your
|
||
* server's available memory by 512 for a rough estimate
|
||
*/
|
||
@IsNumber()
|
||
@IsOptional()
|
||
public WEB_CONCURRENCY = this.toOptionalNumber(process.env.WEB_CONCURRENCY);
|
||
|
||
/**
|
||
* Base64 encoded private key if Outline is to perform SSL termination.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("SSL_CERT")
|
||
public SSL_KEY = this.toOptionalString(process.env.SSL_KEY);
|
||
|
||
/**
|
||
* Base64 encoded public certificate if Outline is to perform SSL termination.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("SSL_KEY")
|
||
public SSL_CERT = this.toOptionalString(process.env.SSL_CERT);
|
||
|
||
/**
|
||
* Should always be left unset in a self-hosted environment.
|
||
*/
|
||
@Equals("hosted")
|
||
@IsOptional()
|
||
public DEPLOYMENT = this.toOptionalString(process.env.DEPLOYMENT);
|
||
|
||
/**
|
||
* Custom company logo that displays on the authentication screen.
|
||
*/
|
||
public TEAM_LOGO = process.env.TEAM_LOGO;
|
||
|
||
/**
|
||
* The default interface language. See translate.getoutline.com for a list of
|
||
* available language codes and their percentage translated.
|
||
*/
|
||
@IsIn(languages)
|
||
public DEFAULT_LANGUAGE = process.env.DEFAULT_LANGUAGE ?? "en_US";
|
||
|
||
/**
|
||
* A comma separated list of which services should be enabled on this
|
||
* instance – defaults to all.
|
||
*/
|
||
public SERVICES =
|
||
process.env.SERVICES ?? "collaboration,websockets,worker,web";
|
||
|
||
/**
|
||
* Auto-redirect to https in production. The default is true but you may set
|
||
* to false if you can be sure that SSL is terminated at an external
|
||
* loadbalancer.
|
||
*/
|
||
@IsBoolean()
|
||
public FORCE_HTTPS = this.toBoolean(process.env.FORCE_HTTPS ?? "true");
|
||
|
||
/**
|
||
* Whether to support multiple subdomains in a single instance.
|
||
*/
|
||
@IsBoolean()
|
||
@Deprecated("The community edition of Outline does not support subdomains")
|
||
public SUBDOMAINS_ENABLED = this.toBoolean(
|
||
process.env.SUBDOMAINS_ENABLED ?? "false"
|
||
);
|
||
|
||
/**
|
||
* Should the installation send anonymized statistics to the maintainers.
|
||
* Defaults to true.
|
||
*/
|
||
@IsBoolean()
|
||
public TELEMETRY = this.toBoolean(
|
||
process.env.ENABLE_UPDATES ?? process.env.TELEMETRY ?? "true"
|
||
);
|
||
|
||
/**
|
||
* Because imports can be much larger than regular file attachments and are
|
||
* deleted automatically we allow an optional separate limit on the size of
|
||
* imports.
|
||
*/
|
||
@IsNumber()
|
||
public MAXIMUM_IMPORT_SIZE =
|
||
this.toOptionalNumber(process.env.MAXIMUM_IMPORT_SIZE) ?? 5120000;
|
||
|
||
/**
|
||
* An optional comma separated list of allowed domains.
|
||
*/
|
||
public ALLOWED_DOMAINS =
|
||
process.env.ALLOWED_DOMAINS ?? process.env.GOOGLE_ALLOWED_DOMAINS;
|
||
|
||
// Third-party services
|
||
|
||
/**
|
||
* The host of your SMTP server for enabling emails.
|
||
*/
|
||
public SMTP_HOST = process.env.SMTP_HOST;
|
||
|
||
/**
|
||
* The port of your SMTP server.
|
||
*/
|
||
@IsNumber()
|
||
@IsOptional()
|
||
public SMTP_PORT = this.toOptionalNumber(process.env.SMTP_PORT);
|
||
|
||
/**
|
||
* The username of your SMTP server, if any.
|
||
*/
|
||
public SMTP_USERNAME = process.env.SMTP_USERNAME;
|
||
|
||
/**
|
||
* The password for the SMTP username, if any.
|
||
*/
|
||
public SMTP_PASSWORD = process.env.SMTP_PASSWORD;
|
||
|
||
/**
|
||
* The email address from which emails are sent.
|
||
*/
|
||
@IsEmail({ allow_display_name: true, allow_ip_domain: true })
|
||
@IsOptional()
|
||
public SMTP_FROM_EMAIL = this.toOptionalString(process.env.SMTP_FROM_EMAIL);
|
||
|
||
/**
|
||
* The reply-to address for emails sent from Outline. If unset the from
|
||
* address is used by default.
|
||
*/
|
||
@IsEmail({ allow_display_name: true, allow_ip_domain: true })
|
||
@IsOptional()
|
||
public SMTP_REPLY_EMAIL = this.toOptionalString(process.env.SMTP_REPLY_EMAIL);
|
||
|
||
/**
|
||
* Override the cipher used for SMTP SSL connections.
|
||
*/
|
||
public SMTP_TLS_CIPHERS = this.toOptionalString(process.env.SMTP_TLS_CIPHERS);
|
||
|
||
/**
|
||
* If true (the default) the connection will use TLS when connecting to server.
|
||
* If false then TLS is used only if server supports the STARTTLS extension.
|
||
*
|
||
* Setting secure to false therefore does not mean that you would not use an
|
||
* encrypted connection.
|
||
*/
|
||
public SMTP_SECURE = this.toBoolean(process.env.SMTP_SECURE ?? "true");
|
||
|
||
/**
|
||
* Sentry DSN for capturing errors and frontend performance.
|
||
*/
|
||
@IsUrl()
|
||
@IsOptional()
|
||
public SENTRY_DSN = this.toOptionalString(process.env.SENTRY_DSN);
|
||
|
||
/**
|
||
* A release SHA or other identifier for Sentry.
|
||
*/
|
||
public RELEASE = this.toOptionalString(process.env.RELEASE);
|
||
|
||
/**
|
||
* An optional host from which to load default avatars.
|
||
*/
|
||
@IsUrl()
|
||
public DEFAULT_AVATAR_HOST =
|
||
process.env.DEFAULT_AVATAR_HOST ?? "https://tiley.herokuapp.com";
|
||
|
||
/**
|
||
* A Google Analytics tracking ID, only v3 supported at this time.
|
||
*/
|
||
@Contains("UA-")
|
||
@IsOptional()
|
||
public GOOGLE_ANALYTICS_ID = this.toOptionalString(
|
||
process.env.GOOGLE_ANALYTICS_ID
|
||
);
|
||
|
||
/**
|
||
* A DataDog API key for tracking server metrics.
|
||
*/
|
||
public DD_API_KEY = process.env.DD_API_KEY;
|
||
|
||
/**
|
||
* Google OAuth2 client credentials. To enable authentication with Google.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("GOOGLE_CLIENT_SECRET")
|
||
public GOOGLE_CLIENT_ID = this.toOptionalString(process.env.GOOGLE_CLIENT_ID);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("GOOGLE_CLIENT_ID")
|
||
public GOOGLE_CLIENT_SECRET = this.toOptionalString(
|
||
process.env.GOOGLE_CLIENT_SECRET
|
||
);
|
||
|
||
/**
|
||
* Slack OAuth2 client credentials. To enable authentication with Slack.
|
||
*/
|
||
@IsOptional()
|
||
@Deprecated("Use SLACK_CLIENT_SECRET instead")
|
||
public SLACK_SECRET = this.toOptionalString(process.env.SLACK_SECRET);
|
||
|
||
@IsOptional()
|
||
@Deprecated("Use SLACK_CLIENT_ID instead")
|
||
public SLACK_KEY = this.toOptionalString(process.env.SLACK_KEY);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("SLACK_CLIENT_SECRET")
|
||
public SLACK_CLIENT_ID = this.toOptionalString(
|
||
process.env.SLACK_CLIENT_ID ?? process.env.SLACK_KEY
|
||
);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("SLACK_CLIENT_ID")
|
||
public SLACK_CLIENT_SECRET = this.toOptionalString(
|
||
process.env.SLACK_CLIENT_SECRET ?? process.env.SLACK_SECRET
|
||
);
|
||
|
||
/**
|
||
* This is injected into the HTML page headers for Slack.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("SLACK_CLIENT_ID")
|
||
public SLACK_VERIFICATION_TOKEN = this.toOptionalString(
|
||
process.env.SLACK_VERIFICATION_TOKEN
|
||
);
|
||
|
||
/**
|
||
* This is injected into the slack-app-id header meta tag if provided.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("SLACK_CLIENT_ID")
|
||
public SLACK_APP_ID = this.toOptionalString(process.env.SLACK_APP_ID);
|
||
|
||
/**
|
||
* If enabled a "Post to Channel" button will be added to search result
|
||
* messages inside of Slack. This also requires setup in Slack UI.
|
||
*/
|
||
@IsOptional()
|
||
@IsBoolean()
|
||
public SLACK_MESSAGE_ACTIONS = this.toBoolean(
|
||
process.env.SLACK_MESSAGE_ACTIONS ?? "false"
|
||
);
|
||
|
||
/**
|
||
* Azure OAuth2 client credentials. To enable authentication with Azure.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("AZURE_CLIENT_SECRET")
|
||
public AZURE_CLIENT_ID = this.toOptionalString(process.env.AZURE_CLIENT_ID);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("AZURE_CLIENT_ID")
|
||
public AZURE_CLIENT_SECRET = this.toOptionalString(
|
||
process.env.AZURE_CLIENT_SECRET
|
||
);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("AZURE_CLIENT_ID")
|
||
public AZURE_RESOURCE_APP_ID = this.toOptionalString(
|
||
process.env.AZURE_RESOURCE_APP_ID
|
||
);
|
||
|
||
/**
|
||
* OICD client credentials. To enable authentication with any
|
||
* compatible provider.
|
||
*/
|
||
@IsOptional()
|
||
@CannotUseWithout("OIDC_CLIENT_SECRET")
|
||
@CannotUseWithout("OIDC_AUTH_URI")
|
||
@CannotUseWithout("OIDC_TOKEN_URI")
|
||
@CannotUseWithout("OIDC_USERINFO_URI")
|
||
@CannotUseWithout("OIDC_DISPLAY_NAME")
|
||
public OIDC_CLIENT_ID = this.toOptionalString(process.env.OIDC_CLIENT_ID);
|
||
|
||
@IsOptional()
|
||
@CannotUseWithout("OIDC_CLIENT_ID")
|
||
public OIDC_CLIENT_SECRET = this.toOptionalString(
|
||
process.env.OIDC_CLIENT_SECRET
|
||
);
|
||
|
||
/**
|
||
* The name of the OIDC provider, eg "GitLab" – this will be displayed on the
|
||
* sign-in button and other places in the UI. The default value is:
|
||
* "OpenID Connect".
|
||
*/
|
||
@MaxLength(50)
|
||
public OIDC_DISPLAY_NAME = process.env.OIDC_DISPLAY_NAME ?? "OpenID Connect";
|
||
|
||
/**
|
||
* The OIDC authorization endpoint.
|
||
*/
|
||
@IsOptional()
|
||
@IsUrl({
|
||
require_tld: false,
|
||
allow_underscores: true,
|
||
})
|
||
public OIDC_AUTH_URI = this.toOptionalString(process.env.OIDC_AUTH_URI);
|
||
|
||
/**
|
||
* The OIDC token endpoint.
|
||
*/
|
||
@IsOptional()
|
||
@IsUrl({
|
||
require_tld: false,
|
||
allow_underscores: true,
|
||
})
|
||
public OIDC_TOKEN_URI = this.toOptionalString(process.env.OIDC_TOKEN_URI);
|
||
|
||
/**
|
||
* The OIDC userinfo endpoint.
|
||
*/
|
||
@IsOptional()
|
||
@IsUrl({
|
||
require_tld: false,
|
||
allow_underscores: true,
|
||
})
|
||
public OIDC_USERINFO_URI = this.toOptionalString(
|
||
process.env.OIDC_USERINFO_URI
|
||
);
|
||
|
||
/**
|
||
* The OIDC profile field to use as the username. The default value is
|
||
* "preferred_username".
|
||
*/
|
||
public OIDC_USERNAME_CLAIM =
|
||
process.env.OIDC_USERNAME_CLAIM ?? "preferred_username";
|
||
|
||
/**
|
||
* A space separated list of OIDC scopes to request. Defaults to "openid
|
||
* profile email".
|
||
*/
|
||
public OIDC_SCOPES = process.env.OIDC_SCOPES ?? "openid profile email";
|
||
|
||
/**
|
||
* A string representing the version of the software.
|
||
*
|
||
* SOURCE_COMMIT is used by Docker Hub
|
||
* SOURCE_VERSION is used by Heroku
|
||
*/
|
||
public VERSION = this.toOptionalString(
|
||
process.env.SOURCE_COMMIT || process.env.SOURCE_VERSION
|
||
);
|
||
|
||
private toOptionalString(value: string | undefined) {
|
||
return value ? value : undefined;
|
||
}
|
||
|
||
private toOptionalNumber(value: string | undefined) {
|
||
return value ? parseInt(value, 10) : undefined;
|
||
}
|
||
|
||
/**
|
||
* Convert a string to a boolean. Supports the following:
|
||
*
|
||
* 0 = false
|
||
* 1 = true
|
||
* "true" = true
|
||
* "false" = false
|
||
* "" = false
|
||
*
|
||
* @param value The string to convert
|
||
* @returns A boolean
|
||
*/
|
||
private toBoolean(value: string) {
|
||
return value ? !!JSON.parse(value) : false;
|
||
}
|
||
}
|
||
|
||
const env = new Environment();
|
||
|
||
export default env;
|