Files
outline/server/test/factories.ts
Tom Moor 10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00

426 lines
9.4 KiB
TypeScript

import { v4 as uuidv4 } from "uuid";
import {
Share,
Team,
User,
Event,
Document,
Star,
Collection,
Group,
GroupUser,
Attachment,
IntegrationAuthentication,
Integration,
AuthenticationProvider,
FileOperation,
WebhookSubscription,
WebhookDelivery,
} from "@server/models";
import {
FileOperationState,
FileOperationType,
} from "@server/models/FileOperation";
let count = 1;
export async function buildShare(overrides: Partial<Share> = {}) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
if (!overrides.documentId) {
const document = await buildDocument({
createdById: overrides.userId,
teamId: overrides.teamId,
});
overrides.documentId = document.id;
}
return Share.create({
published: true,
...overrides,
});
}
export async function buildStar(overrides: Partial<Star> = {}) {
let user;
if (overrides.userId) {
user = await User.findByPk(overrides.userId);
} else {
user = await buildUser();
overrides.userId = user.id;
}
if (!overrides.documentId) {
const document = await buildDocument({
createdById: overrides.userId,
teamId: user?.teamId,
});
overrides.documentId = document.id;
}
return Star.create({
index: "h",
...overrides,
});
}
export function buildTeam(overrides: Record<string, any> = {}) {
count++;
return Team.create(
{
name: `Team ${count}`,
collaborativeEditing: false,
authenticationProviders: [
{
name: "slack",
providerId: uuidv4(),
},
],
...overrides,
},
{
include: "authenticationProviders",
}
);
}
export function buildEvent(overrides: Partial<Event> = {}) {
return Event.create({
name: "documents.publish",
ip: "127.0.0.1",
...overrides,
});
}
export async function buildGuestUser(overrides: Partial<User> = {}) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
count++;
return User.create({
email: `user${count}@example.com`,
name: `User ${count}`,
createdAt: new Date("2018-01-01T00:00:00.000Z"),
lastActiveAt: new Date("2018-01-01T00:00:00.000Z"),
...overrides,
});
}
export async function buildUser(overrides: Partial<User> = {}) {
let team;
if (!overrides.teamId) {
team = await buildTeam();
overrides.teamId = team.id;
} else {
team = await Team.findByPk(overrides.teamId);
}
const authenticationProvider = await AuthenticationProvider.findOne({
where: {
teamId: overrides.teamId,
},
});
count++;
return User.create(
{
email: `user${count}@example.com`,
name: `User ${count}`,
username: `user${count}`,
createdAt: new Date("2018-01-01T00:00:00.000Z"),
updatedAt: new Date("2018-01-02T00:00:00.000Z"),
lastActiveAt: new Date("2018-01-03T00:00:00.000Z"),
authentications: [
{
authenticationProviderId: authenticationProvider!.id,
providerId: uuidv4(),
},
],
...overrides,
},
{
include: "authentications",
}
);
}
export async function buildAdmin(overrides: Partial<User> = {}) {
return buildUser({ ...overrides, isAdmin: true });
}
export async function buildInvite(overrides: Partial<User> = {}) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
const actor = await buildUser({ teamId: overrides.teamId });
count++;
return User.create({
email: `user${count}@example.com`,
name: `User ${count}`,
createdAt: new Date("2018-01-01T00:00:00.000Z"),
invitedById: actor.id,
...overrides,
lastActiveAt: null,
});
}
export async function buildIntegration(overrides: Partial<Integration> = {}) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
const user = await buildUser({
teamId: overrides.teamId,
});
const authentication = await IntegrationAuthentication.create({
service: "slack",
userId: user.id,
teamId: user.teamId,
token: "fake-access-token",
scopes: ["example", "scopes", "here"],
});
return Integration.create({
type: "post",
service: "slack",
settings: {
serviceTeamId: "slack_team_id",
},
authenticationId: authentication.id,
...overrides,
});
}
export async function buildCollection(
overrides: Partial<Collection> & { userId?: string } = {}
) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
count++;
return Collection.create({
name: `Test Collection ${count}`,
description: "Test collection description",
createdById: overrides.userId,
permission: "read_write",
...overrides,
});
}
export async function buildGroup(
overrides: Partial<Group> & { userId?: string } = {}
) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
count++;
return Group.create({
name: `Test Group ${count}`,
createdById: overrides.userId,
...overrides,
});
}
export async function buildGroupUser(
overrides: Partial<GroupUser> & { userId?: string; teamId?: string } = {}
) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
count++;
return GroupUser.create({
createdById: overrides.userId,
...overrides,
});
}
export async function buildDocument(
overrides: Partial<Document> & { userId?: string } = {}
) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser();
overrides.userId = user.id;
}
if (!overrides.collectionId) {
const collection = await buildCollection({
teamId: overrides.teamId,
userId: overrides.userId,
});
overrides.collectionId = collection.id;
}
count++;
return Document.create({
title: `Document ${count}`,
text: "This is the text in an example document",
publishedAt: new Date(),
lastModifiedById: overrides.userId,
createdById: overrides.userId,
...overrides,
});
}
export async function buildFileOperation(
overrides: Partial<FileOperation> = {}
) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildAdmin({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
return FileOperation.create({
state: FileOperationState.Creating,
type: FileOperationType.Export,
size: 0,
key: "uploads/key/to/file.zip",
collectionId: null,
url: "https://www.urltos3file.com/file.zip",
...overrides,
});
}
export async function buildAttachment(overrides: Partial<Attachment> = {}) {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.userId) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.userId = user.id;
}
if (!overrides.documentId) {
const document = await buildDocument({
teamId: overrides.teamId,
userId: overrides.userId,
});
overrides.documentId = document.id;
}
count++;
return Attachment.create({
key: `uploads/key/to/file ${count}.png`,
url: `https://redirect.url.com/uploads/key/to/file ${count}.png`,
contentType: "image/png",
size: 100,
acl: "public-read",
createdAt: new Date("2018-01-02T00:00:00.000Z"),
updatedAt: new Date("2018-01-02T00:00:00.000Z"),
...overrides,
});
}
export async function buildWebhookSubscription(
overrides: Partial<WebhookSubscription> = {}
): Promise<WebhookSubscription> {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.createdById) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.createdById = user.id;
}
if (!overrides.name) {
overrides.name = "Test Webhook Subscription";
}
if (!overrides.url) {
overrides.url = "https://www.example.com/webhook";
}
if (!overrides.events) {
overrides.events = ["*"];
}
if (!overrides.enabled) {
overrides.enabled = true;
}
return WebhookSubscription.create(overrides);
}
export async function buildWebhookDelivery(
overrides: Partial<WebhookDelivery> = {}
): Promise<WebhookDelivery> {
if (!overrides.status) {
overrides.status = "success";
}
if (!overrides.statusCode) {
overrides.statusCode = 200;
}
if (!overrides.requestBody) {
overrides.requestBody = "{}";
}
if (!overrides.requestHeaders) {
overrides.requestHeaders = {};
}
if (!overrides.webhookSubscriptionId) {
const webhookSubscription = await buildWebhookSubscription();
overrides.webhookSubscriptionId = webhookSubscription.id;
}
if (!overrides.createdAt) {
overrides.createdAt = new Date();
}
return WebhookDelivery.create(overrides);
}