feat: Webhooks (#3691)
* Webhooks (#3607) * Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample * WIP: Start getting a Webhook page created. Just the skeleton state right now * WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now * WIP: Get library installed and make TS happy * Get a few checkboxes ready to go * Get creating and destroying working with a decent start to a frontend * Didn't mean to enable this * Remove eslint and fix other random typescript issue * Rename some events to be more realistic * Revert these changes * PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id * Allow any admin on the team to edit webhooks * Start sending some webhooks for some User events * Make sure the URL is valid * Start recording webhook deliveries * Make sure to verify if the subscription is for the type of event we are looking at * Refactor sending Webhooks and follow better webhook schema This creates a presenter to unify the format of webhooks. We also extract the sending of webhooks and recording their deliveries to a method than can be used by each of the different event type methods We also add a status to WebhookDelivery since we need to save the record before we make the HTTP request to get its id. Then once we make the request and get a response we can update the delivery with the HTTP info * Turn off a subscription that has failed for the last 25 deliveries * Get a first spec passing. Found a bug in my returning of promises so good to patch that up now * This looks nicer * Get some tests added for the processor * Add cron task to delete older webhooks * Add Document Events to the Processor * Revisions, FileOperations and Collections * Get all the server side events added to the processor and make Typescript make sure they are all accounted for * Get all the events added to the Frontend and work on styling them a bit, still needs some love though * Get UI styled up a bit * Get events wired up for webhook subscriptions * Get delete events working and test at least one variant of them * Get deletes working and actually make sure to send the model id in the webhook * Remove webhook secrets from this slice * Add disabled label for subscriptions that are disabled * Make sure to cascade the delete * Reorg this file a bit * Fix association * I removed secret for the moment * Apply Copy changes from PR Review Co-authored-by: Tom Moor <tom.moor@gmail.com> * Actually apply the copy changes TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github Co-authored-by: Tom Moor <tom.moor@gmail.com> * Update app/scenes/Settings/Webhooks.tsx Missed this copy change before Co-authored-by: Tom Moor <tom.moor@gmail.com> * Add disabled as yellow badge * Resolve frontend comments * Fixup Schema a bit and remove the dependency on the subscription * Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes * Fix WebhookDeliveries to store the responses as Text instead of blobs * Switch to text better for response bodies, this is using the helpers better and makes the code read better * Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next * Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline * Allow any team admin to see any webhook subscription for the team * Add the indexes based on our lookup patterns * Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes * Allow subscriptions to be edited after creation * Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here * I think this will get these into the translations file * Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy Co-authored-by: Tom Moor <tom.moor@gmail.com> * fix: tsc fix: Document model payload empty * fix: Revision webhook payload Add custom UA for hooks * Add webhooks icon, move under Integrations settings Some spacing fixes * Add actorId to webhook payloads * Add View and ApiKey event types * Spacing tweaks, fix team payload * fix: Webhook not disabled after 25 failures * fix: Enable webhook when editing if previously disabled * fix: Correctly store response headers * fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery * fix: Awkward payload for users.invite webhook * Add BaseEvent, ShareEvent * fix: Add share events to form * fix: Move webhook delivery cleanup to single DB call Remove some unused abstraction * Add user, collection, group context to membership webhook events Some associated refactoring Co-authored-by: Corey Alexander <coreyja@gmail.com>
This commit is contained in:
Tom Moor
GitHub
389
server/types.ts
389
server/types.ts
@@ -1,5 +1,5 @@
|
||||
import { Context } from "koa";
|
||||
import { FileOperation, User } from "./models";
|
||||
import { FileOperation, Team, User } from "./models";
|
||||
|
||||
export type ContextWithState = Context & {
|
||||
state: {
|
||||
@@ -9,227 +9,241 @@ export type ContextWithState = Context & {
|
||||
};
|
||||
};
|
||||
|
||||
export type UserEvent =
|
||||
| {
|
||||
name: "users.create" // eslint-disable-line
|
||||
| "users.signin"
|
||||
| "users.signout"
|
||||
| "users.update"
|
||||
| "users.suspend"
|
||||
| "users.activate"
|
||||
| "users.delete";
|
||||
userId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "users.invite";
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
email: string;
|
||||
name: string;
|
||||
};
|
||||
ip: string;
|
||||
};
|
||||
type BaseEvent = {
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
};
|
||||
|
||||
export type DocumentEvent =
|
||||
| {
|
||||
name: "documents.create" // eslint-disable-line
|
||||
| "documents.publish"
|
||||
| "documents.unpublish"
|
||||
| "documents.delete"
|
||||
| "documents.permanent_delete"
|
||||
| "documents.archive"
|
||||
| "documents.unarchive"
|
||||
| "documents.restore"
|
||||
| "documents.star"
|
||||
| "documents.unstar";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
data: {
|
||||
title: string;
|
||||
source?: "import";
|
||||
};
|
||||
}
|
||||
| {
|
||||
name: "documents.move";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
collectionIds: string[];
|
||||
documentIds: string[];
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "documents.update" // eslint-disable-line
|
||||
| "documents.update.delayed"
|
||||
| "documents.update.debounced";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
createdAt: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
title: string;
|
||||
autosave: boolean;
|
||||
done: boolean;
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "documents.title_change";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
createdAt: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
title: string;
|
||||
previousTitle: string;
|
||||
};
|
||||
ip: string;
|
||||
};
|
||||
export type ApiKeyEvent = BaseEvent & {
|
||||
name: "api_keys.create" | "api_keys.delete";
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type RevisionEvent = {
|
||||
export type UserEvent = BaseEvent &
|
||||
(
|
||||
| {
|
||||
name:
|
||||
| "users.create"
|
||||
| "users.signin"
|
||||
| "users.signout"
|
||||
| "users.update"
|
||||
| "users.suspend"
|
||||
| "users.activate"
|
||||
| "users.delete";
|
||||
userId: string;
|
||||
}
|
||||
| {
|
||||
name: "users.invite";
|
||||
userId: string;
|
||||
data: {
|
||||
email: string;
|
||||
name: string;
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
export type DocumentEvent = BaseEvent &
|
||||
(
|
||||
| {
|
||||
name:
|
||||
| "documents.create"
|
||||
| "documents.publish"
|
||||
| "documents.unpublish"
|
||||
| "documents.delete"
|
||||
| "documents.permanent_delete"
|
||||
| "documents.archive"
|
||||
| "documents.unarchive"
|
||||
| "documents.restore"
|
||||
| "documents.star"
|
||||
| "documents.unstar";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
data: {
|
||||
title: string;
|
||||
source?: "import";
|
||||
};
|
||||
}
|
||||
| {
|
||||
name: "documents.move";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
data: {
|
||||
collectionIds: string[];
|
||||
documentIds: string[];
|
||||
};
|
||||
}
|
||||
| {
|
||||
name:
|
||||
| "documents.update"
|
||||
| "documents.update.delayed"
|
||||
| "documents.update.debounced";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
createdAt: string;
|
||||
data: {
|
||||
title: string;
|
||||
autosave: boolean;
|
||||
done: boolean;
|
||||
};
|
||||
}
|
||||
| {
|
||||
name: "documents.title_change";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
createdAt: string;
|
||||
data: {
|
||||
title: string;
|
||||
previousTitle: string;
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
export type RevisionEvent = BaseEvent & {
|
||||
name: "revisions.create";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
modelId: string;
|
||||
};
|
||||
|
||||
export type FileOperationEvent = {
|
||||
export type FileOperationEvent = BaseEvent & {
|
||||
name:
|
||||
| "fileOperations.create"
|
||||
| "fileOperations.update"
|
||||
| "fileOperation.delete";
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
modelId: string;
|
||||
data: Partial<FileOperation>;
|
||||
};
|
||||
|
||||
export type CollectionEvent =
|
||||
| {
|
||||
name: "collections.create" // eslint-disable-line
|
||||
| "collections.update"
|
||||
| "collections.delete";
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "collections.add_user" | "collections.remove_user";
|
||||
userId: string;
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "collections.add_group" | "collections.remove_group";
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "collections.move";
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
index: string;
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "collections.permission_changed";
|
||||
collectionId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: {
|
||||
privacyChanged: boolean;
|
||||
sharingChanged: boolean;
|
||||
};
|
||||
ip: string;
|
||||
};
|
||||
export type CollectionUserEvent = BaseEvent & {
|
||||
name: "collections.add_user" | "collections.remove_user";
|
||||
userId: string;
|
||||
collectionId: string;
|
||||
};
|
||||
|
||||
export type GroupEvent =
|
||||
| {
|
||||
name: "groups.create" | "groups.delete" | "groups.update";
|
||||
actorId: string;
|
||||
modelId: string;
|
||||
teamId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
ip: string;
|
||||
}
|
||||
| {
|
||||
name: "groups.add_user" | "groups.remove_user";
|
||||
actorId: string;
|
||||
userId: string;
|
||||
modelId: string;
|
||||
teamId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
ip: string;
|
||||
};
|
||||
export type CollectionGroupEvent = BaseEvent & {
|
||||
name: "collections.add_group" | "collections.remove_group";
|
||||
collectionId: string;
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type IntegrationEvent = {
|
||||
export type CollectionEvent = BaseEvent &
|
||||
(
|
||||
| CollectionUserEvent
|
||||
| CollectionGroupEvent
|
||||
| {
|
||||
name:
|
||||
| "collections.create"
|
||||
| "collections.update"
|
||||
| "collections.delete";
|
||||
collectionId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
}
|
||||
| {
|
||||
name: "collections.move";
|
||||
collectionId: string;
|
||||
data: {
|
||||
index: string;
|
||||
};
|
||||
}
|
||||
| {
|
||||
name: "collections.permission_changed";
|
||||
collectionId: string;
|
||||
data: {
|
||||
privacyChanged: boolean;
|
||||
sharingChanged: boolean;
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
export type GroupUserEvent = BaseEvent & {
|
||||
name: "groups.add_user" | "groups.remove_user";
|
||||
userId: string;
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type GroupEvent = BaseEvent &
|
||||
(
|
||||
| GroupUserEvent
|
||||
| {
|
||||
name: "groups.create" | "groups.delete" | "groups.update";
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
export type IntegrationEvent = BaseEvent & {
|
||||
name: "integrations.create" | "integrations.update";
|
||||
modelId: string;
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
};
|
||||
|
||||
export type TeamEvent = {
|
||||
export type TeamEvent = BaseEvent & {
|
||||
name: "teams.update";
|
||||
teamId: string;
|
||||
actorId: string;
|
||||
data: Record<string, any>;
|
||||
ip: string;
|
||||
data: Partial<Team>;
|
||||
};
|
||||
|
||||
export type PinEvent = {
|
||||
export type PinEvent = BaseEvent & {
|
||||
name: "pins.create" | "pins.update" | "pins.delete";
|
||||
teamId: string;
|
||||
modelId: string;
|
||||
documentId: string;
|
||||
collectionId?: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
};
|
||||
|
||||
export type StarEvent = {
|
||||
export type StarEvent = BaseEvent & {
|
||||
name: "stars.create" | "stars.update" | "stars.delete";
|
||||
teamId: string;
|
||||
modelId: string;
|
||||
documentId: string;
|
||||
userId: string;
|
||||
actorId: string;
|
||||
ip: string;
|
||||
};
|
||||
|
||||
export type ShareEvent = BaseEvent & {
|
||||
name: "shares.create" | "shares.update" | "shares.revoke";
|
||||
modelId: string;
|
||||
documentId: string;
|
||||
collectionId?: string;
|
||||
data: {
|
||||
name: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type ViewEvent = BaseEvent & {
|
||||
name: "views.create";
|
||||
documentId: string;
|
||||
collectionId: string;
|
||||
modelId: string;
|
||||
data: {
|
||||
title: string;
|
||||
};
|
||||
};
|
||||
|
||||
export type WebhookSubscriptionEvent = BaseEvent & {
|
||||
name:
|
||||
| "webhook_subscriptions.create"
|
||||
| "webhook_subscriptions.delete"
|
||||
| "webhook_subscriptions.update";
|
||||
modelId: string;
|
||||
data: {
|
||||
name: string;
|
||||
url: string;
|
||||
events: string[];
|
||||
};
|
||||
};
|
||||
|
||||
export type Event =
|
||||
| ApiKeyEvent
|
||||
| UserEvent
|
||||
| DocumentEvent
|
||||
| PinEvent
|
||||
@@ -239,4 +253,7 @@ export type Event =
|
||||
| IntegrationEvent
|
||||
| GroupEvent
|
||||
| RevisionEvent
|
||||
| TeamEvent;
|
||||
| ShareEvent
|
||||
| TeamEvent
|
||||
| ViewEvent
|
||||
| WebhookSubscriptionEvent;
|
||||
|
||||
Reference in New Issue
Block a user