Files
outline/server/env.ts
Tom Moor 10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00

526 lines
14 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/* eslint-disable @typescript-eslint/no-var-requires */
// Load the process environment variables
require("dotenv").config({
silent: true,
});
import {
validate,
IsNotEmpty,
IsUrl,
IsOptional,
IsByteLength,
Equals,
IsNumber,
IsIn,
IsEmail,
IsBoolean,
Contains,
MaxLength,
} from "class-validator";
import { languages } from "@shared/i18n";
import { CannotUseWithout } from "@server/utils/validators";
import Deprecated from "./models/decorators/Deprecated";
export class Environment {
private validationPromise;
constructor() {
this.validationPromise = validate(this);
}
/**
* Allows waiting on the environment to be validated.
*
* @returns A promise that resolves when the environment is validated.
*/
public validate() {
return this.validationPromise;
}
/**
* The current envionment name.
*/
@IsIn(["development", "production", "staging", "test"])
public ENVIRONMENT = process.env.NODE_ENV ?? "production";
/**
* The secret key is used for encrypting data. Do not change this value once
* set or your users will be unable to login.
*/
@IsByteLength(32, 64)
public SECRET_KEY = `${process.env.SECRET_KEY}`;
/**
* The secret that should be passed to the cron utility endpoint to enable
* triggering of scheduled tasks.
*/
@IsNotEmpty()
public UTILS_SECRET = `${process.env.UTILS_SECRET}`;
/**
* The url of the database.
*/
@IsNotEmpty()
@IsUrl({
require_tld: false,
allow_underscores: true,
protocols: ["postgres", "postgresql"],
})
public DATABASE_URL = `${process.env.DATABASE_URL}`;
/**
* The url of the database pool.
*/
@IsOptional()
@IsUrl({
require_tld: false,
allow_underscores: true,
protocols: ["postgres", "postgresql"],
})
public DATABASE_CONNECTION_POOL_URL = this.toOptionalString(
process.env.DATABASE_CONNECTION_POOL_URL
);
/**
* Database connection pool configuration.
*/
@IsNumber()
@IsOptional()
public DATABASE_CONNECTION_POOL_MIN = this.toOptionalNumber(
process.env.DATABASE_CONNECTION_POOL_MIN
);
/**
* Database connection pool configuration.
*/
@IsNumber()
@IsOptional()
public DATABASE_CONNECTION_POOL_MAX = this.toOptionalNumber(
process.env.DATABASE_CONNECTION_POOL_MAX
);
/**
* Set to "disable" to disable SSL connection to the database. This option is
* passed through to Postgres. See:
*
* https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-SSLMODE
*/
@IsIn(["disable", "allow", "require", "prefer", "verify-ca", "verify-full"])
@IsOptional()
public PGSSLMODE = process.env.PGSSLMODE;
/**
* The url of redis. Note that redis does not have a database after the port.
* Note: More extensive validation isn't included here due to our support for
* base64-encoded configuration.
*/
@IsNotEmpty()
public REDIS_URL = process.env.REDIS_URL;
/**
* The fully qualified, external facing domain name of the server.
*/
@IsNotEmpty()
@IsUrl({ require_tld: false })
public URL = `${process.env.URL}`;
/**
* If using a Cloudfront/Cloudflare distribution or similar it can be set below.
* This will cause paths to javascript, stylesheets, and images to be updated to
* the hostname defined in CDN_URL. In your CDN configuration the origin server
* should be set to the same as URL.
*/
@IsOptional()
@IsUrl()
public CDN_URL = this.toOptionalString(process.env.CDN_URL);
/**
* The fully qualified, external facing domain name of the collaboration
* service, if different (unlikely)
*/
@IsUrl({ require_tld: false, protocols: ["http", "https", "ws", "wss"] })
@IsOptional()
public COLLABORATION_URL = this.toOptionalString(
process.env.COLLABORATION_URL
);
/**
* The port that the server will listen on, defaults to 3000.
*/
@IsNumber()
@IsOptional()
public PORT = this.toOptionalNumber(process.env.PORT);
/**
* Optional extra debugging. Comma separated
*/
public DEBUG = `${process.env.DEBUG}`;
/**
* How many processes should be spawned. As a reasonable rule divide your
* server's available memory by 512 for a rough estimate
*/
@IsNumber()
@IsOptional()
public WEB_CONCURRENCY = this.toOptionalNumber(process.env.WEB_CONCURRENCY);
/**
* Base64 encoded private key if Outline is to perform SSL termination.
*/
@IsOptional()
@CannotUseWithout("SSL_CERT")
public SSL_KEY = this.toOptionalString(process.env.SSL_KEY);
/**
* Base64 encoded public certificate if Outline is to perform SSL termination.
*/
@IsOptional()
@CannotUseWithout("SSL_KEY")
public SSL_CERT = this.toOptionalString(process.env.SSL_CERT);
/**
* Should always be left unset in a self-hosted environment.
*/
@Equals("hosted")
@IsOptional()
public DEPLOYMENT = this.toOptionalString(process.env.DEPLOYMENT);
/**
* Custom company logo that displays on the authentication screen.
*/
public TEAM_LOGO = process.env.TEAM_LOGO;
/**
* The default interface language. See translate.getoutline.com for a list of
* available language codes and their percentage translated.
*/
@IsIn(languages)
public DEFAULT_LANGUAGE = process.env.DEFAULT_LANGUAGE ?? "en_US";
/**
* A comma separated list of which services should be enabled on this
* instance defaults to all.
*/
public SERVICES =
process.env.SERVICES ?? "collaboration,websockets,worker,web";
/**
* Auto-redirect to https in production. The default is true but you may set
* to false if you can be sure that SSL is terminated at an external
* loadbalancer.
*/
@IsBoolean()
public FORCE_HTTPS = this.toBoolean(process.env.FORCE_HTTPS ?? "true");
/**
* Whether to support multiple subdomains in a single instance.
*/
@IsBoolean()
@Deprecated("The community edition of Outline does not support subdomains")
public SUBDOMAINS_ENABLED = this.toBoolean(
process.env.SUBDOMAINS_ENABLED ?? "false"
);
/**
* Should the installation send anonymized statistics to the maintainers.
* Defaults to true.
*/
@IsBoolean()
public TELEMETRY = this.toBoolean(
process.env.ENABLE_UPDATES ?? process.env.TELEMETRY ?? "true"
);
/**
* Because imports can be much larger than regular file attachments and are
* deleted automatically we allow an optional separate limit on the size of
* imports.
*/
@IsNumber()
public MAXIMUM_IMPORT_SIZE =
this.toOptionalNumber(process.env.MAXIMUM_IMPORT_SIZE) ?? 5120000;
/**
* An optional comma separated list of allowed domains.
*/
public ALLOWED_DOMAINS =
process.env.ALLOWED_DOMAINS ?? process.env.GOOGLE_ALLOWED_DOMAINS;
// Third-party services
/**
* The host of your SMTP server for enabling emails.
*/
public SMTP_HOST = process.env.SMTP_HOST;
/**
* The port of your SMTP server.
*/
@IsNumber()
@IsOptional()
public SMTP_PORT = this.toOptionalNumber(process.env.SMTP_PORT);
/**
* The username of your SMTP server, if any.
*/
public SMTP_USERNAME = process.env.SMTP_USERNAME;
/**
* The password for the SMTP username, if any.
*/
public SMTP_PASSWORD = process.env.SMTP_PASSWORD;
/**
* The email address from which emails are sent.
*/
@IsEmail({ allow_display_name: true, allow_ip_domain: true })
@IsOptional()
public SMTP_FROM_EMAIL = this.toOptionalString(process.env.SMTP_FROM_EMAIL);
/**
* The reply-to address for emails sent from Outline. If unset the from
* address is used by default.
*/
@IsEmail({ allow_display_name: true, allow_ip_domain: true })
@IsOptional()
public SMTP_REPLY_EMAIL = this.toOptionalString(process.env.SMTP_REPLY_EMAIL);
/**
* Override the cipher used for SMTP SSL connections.
*/
public SMTP_TLS_CIPHERS = this.toOptionalString(process.env.SMTP_TLS_CIPHERS);
/**
* If true (the default) the connection will use TLS when connecting to server.
* If false then TLS is used only if server supports the STARTTLS extension.
*
* Setting secure to false therefore does not mean that you would not use an
* encrypted connection.
*/
public SMTP_SECURE = this.toBoolean(process.env.SMTP_SECURE ?? "true");
/**
* Sentry DSN for capturing errors and frontend performance.
*/
@IsUrl()
@IsOptional()
public SENTRY_DSN = this.toOptionalString(process.env.SENTRY_DSN);
/**
* A release SHA or other identifier for Sentry.
*/
public RELEASE = this.toOptionalString(process.env.RELEASE);
/**
* An optional host from which to load default avatars.
*/
@IsUrl()
public DEFAULT_AVATAR_HOST =
process.env.DEFAULT_AVATAR_HOST ?? "https://tiley.herokuapp.com";
/**
* A Google Analytics tracking ID, only v3 supported at this time.
*/
@Contains("UA-")
@IsOptional()
public GOOGLE_ANALYTICS_ID = this.toOptionalString(
process.env.GOOGLE_ANALYTICS_ID
);
/**
* A DataDog API key for tracking server metrics.
*/
public DD_API_KEY = process.env.DD_API_KEY;
/**
* Google OAuth2 client credentials. To enable authentication with Google.
*/
@IsOptional()
@CannotUseWithout("GOOGLE_CLIENT_SECRET")
public GOOGLE_CLIENT_ID = this.toOptionalString(process.env.GOOGLE_CLIENT_ID);
@IsOptional()
@CannotUseWithout("GOOGLE_CLIENT_ID")
public GOOGLE_CLIENT_SECRET = this.toOptionalString(
process.env.GOOGLE_CLIENT_SECRET
);
/**
* Slack OAuth2 client credentials. To enable authentication with Slack.
*/
@IsOptional()
@Deprecated("Use SLACK_CLIENT_SECRET instead")
public SLACK_SECRET = this.toOptionalString(process.env.SLACK_SECRET);
@IsOptional()
@Deprecated("Use SLACK_CLIENT_ID instead")
public SLACK_KEY = this.toOptionalString(process.env.SLACK_KEY);
@IsOptional()
@CannotUseWithout("SLACK_CLIENT_SECRET")
public SLACK_CLIENT_ID = this.toOptionalString(
process.env.SLACK_CLIENT_ID ?? process.env.SLACK_KEY
);
@IsOptional()
@CannotUseWithout("SLACK_CLIENT_ID")
public SLACK_CLIENT_SECRET = this.toOptionalString(
process.env.SLACK_CLIENT_SECRET ?? process.env.SLACK_SECRET
);
/**
* This is injected into the HTML page headers for Slack.
*/
@IsOptional()
@CannotUseWithout("SLACK_CLIENT_ID")
public SLACK_VERIFICATION_TOKEN = this.toOptionalString(
process.env.SLACK_VERIFICATION_TOKEN
);
/**
* This is injected into the slack-app-id header meta tag if provided.
*/
@IsOptional()
@CannotUseWithout("SLACK_CLIENT_ID")
public SLACK_APP_ID = this.toOptionalString(process.env.SLACK_APP_ID);
/**
* If enabled a "Post to Channel" button will be added to search result
* messages inside of Slack. This also requires setup in Slack UI.
*/
@IsOptional()
@IsBoolean()
public SLACK_MESSAGE_ACTIONS = this.toBoolean(
process.env.SLACK_MESSAGE_ACTIONS ?? "false"
);
/**
* Azure OAuth2 client credentials. To enable authentication with Azure.
*/
@IsOptional()
@CannotUseWithout("AZURE_CLIENT_SECRET")
public AZURE_CLIENT_ID = this.toOptionalString(process.env.AZURE_CLIENT_ID);
@IsOptional()
@CannotUseWithout("AZURE_CLIENT_ID")
public AZURE_CLIENT_SECRET = this.toOptionalString(
process.env.AZURE_CLIENT_SECRET
);
@IsOptional()
@CannotUseWithout("AZURE_CLIENT_ID")
public AZURE_RESOURCE_APP_ID = this.toOptionalString(
process.env.AZURE_RESOURCE_APP_ID
);
/**
* OICD client credentials. To enable authentication with any
* compatible provider.
*/
@IsOptional()
@CannotUseWithout("OIDC_CLIENT_SECRET")
@CannotUseWithout("OIDC_AUTH_URI")
@CannotUseWithout("OIDC_TOKEN_URI")
@CannotUseWithout("OIDC_USERINFO_URI")
@CannotUseWithout("OIDC_DISPLAY_NAME")
public OIDC_CLIENT_ID = this.toOptionalString(process.env.OIDC_CLIENT_ID);
@IsOptional()
@CannotUseWithout("OIDC_CLIENT_ID")
public OIDC_CLIENT_SECRET = this.toOptionalString(
process.env.OIDC_CLIENT_SECRET
);
/**
* The name of the OIDC provider, eg "GitLab" this will be displayed on the
* sign-in button and other places in the UI. The default value is:
* "OpenID Connect".
*/
@MaxLength(50)
public OIDC_DISPLAY_NAME = process.env.OIDC_DISPLAY_NAME ?? "OpenID Connect";
/**
* The OIDC authorization endpoint.
*/
@IsOptional()
@IsUrl({
require_tld: false,
allow_underscores: true,
})
public OIDC_AUTH_URI = this.toOptionalString(process.env.OIDC_AUTH_URI);
/**
* The OIDC token endpoint.
*/
@IsOptional()
@IsUrl({
require_tld: false,
allow_underscores: true,
})
public OIDC_TOKEN_URI = this.toOptionalString(process.env.OIDC_TOKEN_URI);
/**
* The OIDC userinfo endpoint.
*/
@IsOptional()
@IsUrl({
require_tld: false,
allow_underscores: true,
})
public OIDC_USERINFO_URI = this.toOptionalString(
process.env.OIDC_USERINFO_URI
);
/**
* The OIDC profile field to use as the username. The default value is
* "preferred_username".
*/
public OIDC_USERNAME_CLAIM =
process.env.OIDC_USERNAME_CLAIM ?? "preferred_username";
/**
* A space separated list of OIDC scopes to request. Defaults to "openid
* profile email".
*/
public OIDC_SCOPES = process.env.OIDC_SCOPES ?? "openid profile email";
/**
* A string representing the version of the software.
*
* SOURCE_COMMIT is used by Docker Hub
* SOURCE_VERSION is used by Heroku
*/
public VERSION = this.toOptionalString(
process.env.SOURCE_COMMIT || process.env.SOURCE_VERSION
);
private toOptionalString(value: string | undefined) {
return value ? value : undefined;
}
private toOptionalNumber(value: string | undefined) {
return value ? parseInt(value, 10) : undefined;
}
/**
* Convert a string to a boolean. Supports the following:
*
* 0 = false
* 1 = true
* "true" = true
* "false" = false
* "" = false
*
* @param value The string to convert
* @returns A boolean
*/
private toBoolean(value: string) {
return value ? !!JSON.parse(value) : false;
}
}
const env = new Environment();
export default env;