yuuza/outline
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: Webhooks (#3691)

Browse Source 
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
This commit is contained in:
Tom Moor
2022-06-29 08:44:50 +03:00
committed by GitHub
parent 9a6e09bafa
commit 10f86ed218
53 changed files with 2531 additions and 247 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
server/commands/userInviter.ts
View File

@@ -73,6 +73,7 @@ export default async function userInviter({
name: "users.invite",
actorId: user.id,
teamId: user.teamId,
userId: newUser.id,
data: {
email: invite.email,
name: invite.name,

10
server/env.ts
View File

@@ -485,6 +485,16 @@ export class Environment {
*/
public OIDC_SCOPES = process.env.OIDC_SCOPES ?? "openid profile email";
/**
* A string representing the version of the software.
*
* SOURCE_COMMIT is used by Docker Hub
* SOURCE_VERSION is used by Heroku
*/
public VERSION = this.toOptionalString(
process.env.SOURCE_COMMIT || process.env.SOURCE_VERSION
);
private toOptionalString(value: string | undefined) {
return value ? value : undefined;
}

4
server/logging/tracing.ts
View File

@@ -8,9 +8,7 @@ export * as APM from "@theo.gravity/datadog-apm";
if (env.DD_API_KEY) {
init(
{
// SOURCE_COMMIT is used by Docker Hub
// SOURCE_VERSION is used by Heroku
version: process.env.SOURCE_COMMIT || process.env.SOURCE_VERSION,
version: env.VERSION,
service: process.env.DD_SERVICE || "outline",
},
{

68
server/migrations/20220521164111-create-webhook-subscription.js Normal file
View File

@@ -0,0 +1,68 @@
"use strict";
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.sequelize.transaction(async (transaction) => {
await queryInterface.createTable(
"webhook_subscriptions",
{
id: {
type: Sequelize.UUID,
allowNull: false,
primaryKey: true,
},
teamId: {
type: Sequelize.UUID,
allowNull: false,
references: {
model: "teams",
},
},
createdById: {
type: Sequelize.UUID,
allowNull: false,
references: {
model: "users",
},
},
url: {
type: Sequelize.STRING,
allowNull: false,
},
enabled: {
type: Sequelize.BOOLEAN,
allowNull: false,
},
name: {
type: Sequelize.STRING,
allowNull: false,
},
events: {
type: Sequelize.ARRAY(Sequelize.STRING),
allowNull: false,
},
createdAt: {
allowNull: false,
type: Sequelize.DATE,
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE,
},
},
{ transaction }
);
await queryInterface.addIndex(
"webhook_subscriptions",
["teamId", "enabled"],
{
name: "webhook_subscriptions_team_id_enabled",
transaction,
}
);
});
},
down: async (queryInterface, Sequelize) => {
return queryInterface.dropTable("webhook_subscriptions");
},
};

73
server/migrations/20220606031139-create-webhook-delivieries.js Normal file
View File

@@ -0,0 +1,73 @@
"use strict";
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.sequelize.transaction(async (transaction) => {
await queryInterface.createTable(
"webhook_deliveries",
{
id: {
type: Sequelize.UUID,
allowNull: false,
primaryKey: true,
},
webhookSubscriptionId: {
type: Sequelize.UUID,
allowNull: false,
onDelete: "cascade",
references: {
model: "webhook_subscriptions",
},
},
status: {
type: Sequelize.STRING,
allowNull: false,
},
statusCode: {
type: Sequelize.INTEGER,
allowNull: true,
},
requestBody: {
type: Sequelize.JSONB,
allowNull: true,
},
requestHeaders: {
type: Sequelize.JSONB,
allowNull: true,
},
responseBody: {
type: Sequelize.TEXT,
allowNull: true,
},
responseHeaders: {
type: Sequelize.JSONB,
allowNull: true,
},
createdAt: {
type: Sequelize.DATE,
allowNull: false,
},
updatedAt: {
type: Sequelize.DATE,
allowNull: false,
},
},
{ transaction }
);
await queryInterface.addIndex(
"webhook_deliveries",
["webhookSubscriptionId"],
{
name: "webhook_deliveries_webhook_subscription_id",
transaction,
}
);
await queryInterface.addIndex("webhook_deliveries", ["createdAt"], {
name: "webhook_deliveries_createdAt",
transaction,
});
});
},
down: async (queryInterface, Sequelize) => {
return queryInterface.dropTable("webhook_deliveries");
},
};

17
server/models/CollectionGroup.ts
View File

@@ -6,6 +6,7 @@ import {
IsIn,
Table,
DataType,
Scopes,
} from "sequelize-typescript";
import Collection from "./Collection";
import Group from "./Group";
@@ -13,6 +14,22 @@ import User from "./User";
import BaseModel from "./base/BaseModel";
import Fix from "./decorators/Fix";
@Scopes(() => ({
withGroup: {
include: [
{
association: "group",
},
],
},
withCollection: {
include: [
{
association: "collection",
},
],
},
}))
@Table({ tableName: "collection_groups", modelName: "collection_group" })
@Fix
class CollectionGroup extends BaseModel {

17
server/models/CollectionUser.ts
View File

@@ -6,12 +6,29 @@ import {
IsIn,
Table,
DataType,
Scopes,
} from "sequelize-typescript";
import Collection from "./Collection";
import User from "./User";
import BaseModel from "./base/BaseModel";
import Fix from "./decorators/Fix";
@Scopes(() => ({
withUser: {
include: [
{
association: "user",
},
],
},
withCollection: {
include: [
{
association: "collection",
},
],
},
}))
@Table({ tableName: "collection_users", modelName: "collection_user" })
@Fix
class CollectionUser extends BaseModel {

2
server/models/Event.ts
View File

@@ -167,6 +167,8 @@ class Event extends IdModel {
"users.suspend",
"users.activate",
"users.delete",
"webhook_subscriptions.create",
"webhook_subscriptions.delete",
];
}

17
server/models/GroupUser.ts
View File

@@ -5,6 +5,7 @@ import {
Column,
Table,
DataType,
Scopes,
} from "sequelize-typescript";
import Group from "./Group";
import User from "./User";
@@ -18,6 +19,22 @@ import Fix from "./decorators/Fix";
},
],
}))
@Scopes(() => ({
withGroup: {
include: [
{
association: "group",
},
],
},
withUser: {
include: [
{
association: "user",
},
],
},
}))
@Table({ tableName: "group_users", modelName: "group_user", paranoid: true })
@Fix
class GroupUser extends BaseModel {

3
server/models/Team.ts
View File

@@ -35,6 +35,9 @@ import Fix from "./decorators/Fix";
const readFile = util.promisify(fs.readFile);
@Scopes(() => ({
withDomains: {
include: [{ model: TeamDomain }],
},
withAuthenticationProviders: {
include: [
{

12
server/models/View.ts
View File

@@ -7,6 +7,7 @@ import {
ForeignKey,
Table,
DataType,
Scopes,
} from "sequelize-typescript";
import { USER_PRESENCE_INTERVAL } from "@shared/constants";
import Document from "./Document";
@@ -14,6 +15,17 @@ import User from "./User";
import IdModel from "./base/IdModel";
import Fix from "./decorators/Fix";
@Scopes(() => ({
withUser: () => ({
include: [
{
model: User,
required: true,
as: "user",
},
],
}),
}))
@Table({ tableName: "views", modelName: "view" })
@Fix
class View extends IdModel {

53
server/models/WebhookDelivery.ts Normal file
View File

@@ -0,0 +1,53 @@
import {
Column,
Table,
BelongsTo,
ForeignKey,
NotEmpty,
DataType,
IsIn,
} from "sequelize-typescript";
import WebhookSubscription from "./WebhookSubscription";
import IdModel from "./base/IdModel";
import Fix from "./decorators/Fix";
@Table({
tableName: "webhook_deliveries",
modelName: "webhook_delivery",
})
@Fix
class WebhookDelivery extends IdModel {
@NotEmpty
@IsIn([["pending", "success", "failed"]])
@Column(DataType.STRING)
status: "pending" | "success" | "failed";
@Column(DataType.INTEGER)
statusCode: number;
@Column(DataType.JSONB)
requestBody: unknown;
@Column(DataType.JSONB)
requestHeaders: Record<string, string>;
@Column(DataType.TEXT)
responseBody: string;
@Column(DataType.JSONB)
responseHeaders: Record<string, string>;
@Column(DataType.DATE)
createdAt: Date;
// associations
@BelongsTo(() => WebhookSubscription, "webhookSubscriptionId")
webhookSubscription: WebhookSubscription;
@ForeignKey(() => WebhookSubscription)
@Column
webhookSubscriptionId: string;
}
export default WebhookDelivery;

70
server/models/WebhookSubscription.ts Normal file
View File

@@ -0,0 +1,70 @@
import { bool } from "aws-sdk/clients/signer";
import {
Column,
Table,
BelongsTo,
ForeignKey,
NotEmpty,
DataType,
IsUrl,
} from "sequelize-typescript";
import { Event } from "@server/types";
import Team from "./Team";
import User from "./User";
import IdModel from "./base/IdModel";
import Fix from "./decorators/Fix";
@Table({
tableName: "webhook_subscriptions",
modelName: "webhook_subscription",
})
@Fix
class WebhookSubscription extends IdModel {
@NotEmpty
@Column
name: string;
@IsUrl
@NotEmpty
@Column
url: string;
@Column
enabled: boolean;
@Column(DataType.ARRAY(DataType.STRING))
events: string[];
// associations
@BelongsTo(() => User, "createdById")
createdBy: User;
@ForeignKey(() => User)
@Column
createdById: string;
@BelongsTo(() => Team, "teamId")
team: Team;
@ForeignKey(() => Team)
@Column
teamId: string;
// methods
validForEvent = (event: Event): bool => {
if (this.events.length === 1 && this.events[0] === "*") {
return true;
}
for (const e of this.events) {
if (e === event.name || event.name.startsWith(e + ".")) {
return true;
}
}
return false;
};
}
export default WebhookSubscription;

4
server/models/index.ts
View File

@@ -49,3 +49,7 @@ export { default as User } from "./User";
export { default as UserAuthentication } from "./UserAuthentication";
export { default as View } from "./View";
export { default as WebhookSubscription } from "./WebhookSubscription";
export { default as WebhookDelivery } from "./WebhookDelivery";

1
server/policies/index.ts
View File

@@ -21,6 +21,7 @@ import "./star";
import "./user";
import "./team";
import "./group";
import "./webhookSubscription";
type Policy = Record<string, boolean>;

35
server/policies/webhookSubscription.ts Normal file
View File

@@ -0,0 +1,35 @@
import { User, Team, WebhookSubscription } from "@server/models";
import { allow } from "./cancan";
allow(User, "listWebhookSubscription", Team, (user, team) => {
if (!team || user.isViewer || user.teamId !== team.id) {
return false;
}
return user.isAdmin;
});
allow(User, "createWebhookSubscription", Team, (user, team) => {
if (!team || user.isViewer || user.teamId !== team.id) {
return false;
}
return user.isAdmin;
});
allow(
User,
["read", "update", "delete"],
WebhookSubscription,
(user, webhook): boolean => {
if (!user || !webhook) {
return false;
}
if (!user.isAdmin) {
return false;
}
return user.teamId === webhook.teamId;
}
);

9
server/presenters/groupMembership.ts
View File

@@ -5,14 +5,17 @@ type GroupMembership = {
id: string;
userId: string;
groupId: string;
user: ReturnType<typeof presentUser>;
user?: ReturnType<typeof presentUser>;
};
export default (membership: GroupUser): GroupMembership => {
export default (
membership: GroupUser,
options?: { includeUser: boolean }
): GroupMembership => {
return {
id: `${membership.userId}-${membership.groupId}`,
userId: membership.userId,
groupId: membership.groupId,
user: presentUser(membership.user),
user: options?.includeUser ? presentUser(membership.user) : undefined,
};
};

4
server/presenters/index.ts
View File

@@ -20,6 +20,8 @@ import presentStar from "./star";
import presentTeam from "./team";
import presentUser from "./user";
import presentView from "./view";
import presentWebhook from "./webhook";
import presentWebhookSubscription from "./webhookSubscription";
export {
presentApiKey,
@@ -44,4 +46,6 @@ export {
presentPolicies,
presentGroupMembership,
presentCollectionGroupMembership,
presentWebhook,
presentWebhookSubscription,
};

5
server/presenters/user.ts
View File

@@ -20,10 +20,7 @@ type UserPresentation = {
language?: string;
};
export default (
user: User,
options: Options = {}
): UserPresentation | null | undefined => {
export default (user: User, options: Options = {}): UserPresentation => {
const userData: UserPresentation = {
id: user.id,
name: user.name,

38
server/presenters/webhook.ts Normal file
View File

@@ -0,0 +1,38 @@
import { WebhookDelivery } from "@server/models";
import { Event } from "@server/types";
export interface WebhookPayload {
model: Record<string, unknown> | null;
id: string;
[key: string]: unknown;
}
interface WebhookProps {
event: Event;
delivery: WebhookDelivery;
payload: WebhookPayload;
}
export interface WebhookPresentation {
id: string;
actorId: string;
webhookSubscriptionId: string;
event: string;
payload: WebhookPayload;
createdAt: Date;
}
export default function present({
event,
delivery,
payload,
}: WebhookProps): WebhookPresentation {
return {
id: delivery.id,
actorId: event.actorId,
webhookSubscriptionId: delivery.webhookSubscriptionId,
createdAt: delivery.createdAt,
event: event.name,
payload: payload,
};
}

13
server/presenters/webhookSubscription.ts Normal file
View File

@@ -0,0 +1,13 @@
import { WebhookSubscription } from "@server/models";
export default function present(webhook: WebhookSubscription) {
return {
id: webhook.id,
name: webhook.name,
url: webhook.url,
events: webhook.events,
enabled: webhook.enabled,
createdAt: webhook.createdAt,
updatedAt: webhook.updatedAt,
};
}

2
server/queues/processors/BaseProcessor.ts
View File

@@ -1,7 +1,7 @@
import { Event } from "@server/types";
export default abstract class BaseProcessor {
static applicableEvents: (Event["name"] | "*")[] = [];
static applicableEvents: Event["name"][] | ["*"] = [];
public abstract perform(event: Event): Promise<void>;
}

9
server/queues/processors/NotificationsProcessor.test.ts
View File

@@ -118,6 +118,9 @@ describe("revisions.create", () => {
documentId: document.id,
collectionId: document.collectionId,
teamId: document.teamId,
actorId: collaborator.id,
modelId: document.id,
ip,
});
expect(DocumentNotificationEmail.schedule).toHaveBeenCalled();
});
@@ -142,6 +145,9 @@ describe("revisions.create", () => {
documentId: document.id,
collectionId: document.collectionId,
teamId: document.teamId,
actorId: collaborator.id,
modelId: document.id,
ip,
});
expect(DocumentNotificationEmail.schedule).not.toHaveBeenCalled();
});
@@ -163,6 +169,9 @@ describe("revisions.create", () => {
documentId: document.id,
collectionId: document.collectionId,
teamId: document.teamId,
actorId: user.id,
modelId: document.id,
ip,
});
expect(DocumentNotificationEmail.schedule).not.toHaveBeenCalled();
});

96
server/queues/processors/WebhookProcessor.test.ts Normal file
View File

@@ -0,0 +1,96 @@
import { buildUser, buildWebhookSubscription } from "@server/test/factories";
import { flushdb } from "@server/test/support";
import { UserEvent } from "@server/types";
import DeliverWebhookTask from "../tasks/DeliverWebhookTask";
import WebhookProcessor from "./WebhookProcessor";
jest.mock("@server/queues/tasks/DeliverWebhookTask");
beforeEach(() => flushdb());
beforeEach(() => {
jest.resetAllMocks();
});
const ip = "127.0.0.1";
describe("WebhookProcessor", () => {
test("it schedules a delivery for the event", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
const signedInUser = await buildUser({ teamId: subscription.teamId });
const processor = new WebhookProcessor();
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await processor.perform(event);
expect(DeliverWebhookTask.schedule).toHaveBeenCalled();
expect(DeliverWebhookTask.schedule).toHaveBeenCalledWith({
event,
subscriptionId: subscription.id,
});
});
test("not schedule a delivery when not subscribed to event", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["users.create"],
});
const signedInUser = await buildUser({ teamId: subscription.teamId });
const processor = new WebhookProcessor();
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await processor.perform(event);
expect(DeliverWebhookTask.schedule).toHaveBeenCalledTimes(0);
});
test("it schedules a delivery for the event for each subscription", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
const subscriptionTwo = await buildWebhookSubscription({
url: "http://example.com",
teamId: subscription.teamId,
events: ["*"],
});
const signedInUser = await buildUser({ teamId: subscription.teamId });
const processor = new WebhookProcessor();
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await processor.perform(event);
expect(DeliverWebhookTask.schedule).toHaveBeenCalled();
expect(DeliverWebhookTask.schedule).toHaveBeenCalledTimes(2);
expect(DeliverWebhookTask.schedule).toHaveBeenCalledWith({
event,
subscriptionId: subscription.id,
});
expect(DeliverWebhookTask.schedule).toHaveBeenCalledWith({
event,
subscriptionId: subscriptionTwo.id,
});
});
});

27
server/queues/processors/WebhookProcessor.ts Normal file
View File

@@ -0,0 +1,27 @@
import { WebhookSubscription } from "@server/models";
import { Event } from "@server/types";
import DeliverWebhookTask from "../tasks/DeliverWebhookTask";
import BaseProcessor from "./BaseProcessor";
export default class WebhookProcessor extends BaseProcessor {
static applicableEvents: ["*"] = ["*"];
async perform(event: Event) {
const webhookSubscriptions = await WebhookSubscription.findAll({
where: {
enabled: true,
teamId: event.teamId,
},
});
const applicableSubscriptions = webhookSubscriptions.filter((webhook) =>
webhook.validForEvent(event)
);
await Promise.all(
applicableSubscriptions.map((subscription) =>
DeliverWebhookTask.schedule({ event, subscriptionId: subscription.id })
)
);
}
}

33
server/queues/tasks/CleanupWebhookDeliveriesTask.test.ts Normal file
View File

@@ -0,0 +1,33 @@
import { subDays } from "date-fns";
import { WebhookDelivery } from "@server/models";
import { buildWebhookDelivery } from "@server/test/factories";
import { flushdb } from "@server/test/support";
import CleanupWebhookDeliveriesTask from "./CleanupWebhookDeliveriesTask";
beforeEach(() => flushdb());
const deliveryExists = async (delivery: WebhookDelivery) => {
const results = await WebhookDelivery.findOne({ where: { id: delivery.id } });
return !!results;
};
describe("CleanupWebookDeliveriesTask", () => {
it("should delete Webhook Deliveries older than 1 week", async () => {
const brandNewWebhookDelivery = await buildWebhookDelivery({
createdAt: new Date(),
});
const newishWebhookDelivery = await buildWebhookDelivery({
createdAt: subDays(new Date(), 5),
});
const oldWebhookDelivery = await buildWebhookDelivery({
createdAt: subDays(new Date(), 8),
});
const task = new CleanupWebhookDeliveriesTask();
await task.perform();
expect(await deliveryExists(brandNewWebhookDelivery)).toBe(true);
expect(await deliveryExists(newishWebhookDelivery)).toBe(true);
expect(await deliveryExists(oldWebhookDelivery)).toBe(false);
});
});

28
server/queues/tasks/CleanupWebhookDeliveriesTask.ts Normal file
View File

@@ -0,0 +1,28 @@
import { subDays } from "date-fns";
import { Op } from "sequelize";
import Logger from "@server/logging/Logger";
import { WebhookDelivery } from "@server/models";
import BaseTask, { TaskPriority } from "./BaseTask";
type Props = void;
export default class CleanupWebhookDeliveriesTask extends BaseTask<Props> {
public async perform(_: Props) {
Logger.info("task", `Deleting WebhookDeliveries older than one week…`);
const count = await WebhookDelivery.unscoped().destroy({
where: {
createdAt: {
[Op.lt]: subDays(new Date(), 7),
},
},
});
Logger.info("task", `${count} old WebhookDeliveries deleted.`);
}
public get options() {
return {
attempts: 1,
priority: TaskPriority.Background,
};
}
}

202
server/queues/tasks/DeliverWebhookTask.test.ts Normal file
View File

@@ -0,0 +1,202 @@
import fetchMock from "jest-fetch-mock";
import { v4 as uuidv4 } from "uuid";
import { WebhookDelivery } from "@server/models";
import {
buildUser,
buildWebhookDelivery,
buildWebhookSubscription,
} from "@server/test/factories";
import { flushdb } from "@server/test/support";
import { UserEvent } from "@server/types";
import DeliverWebhookTask from "./DeliverWebhookTask";
beforeEach(() => flushdb());
beforeEach(() => {
jest.resetAllMocks();
fetchMock.resetMocks();
});
const ip = "127.0.0.1";
fetchMock.enableMocks();
describe("DeliverWebhookTask", () => {
test("should hit the subscription url and record a delivery", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
const signedInUser = await buildUser({ teamId: subscription.teamId });
const processor = new DeliverWebhookTask();
fetchMock.mockResponse("SUCCESS", { status: 200 });
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await processor.perform({
subscriptionId: subscription.id,
event,
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock).toHaveBeenCalledWith(
"http://example.com",
expect.anything()
);
const parsedBody = JSON.parse(
fetchMock.mock.calls[0]![1]!.body!.toString()
);
expect(parsedBody.webhookSubscriptionId).toBe(subscription.id);
expect(parsedBody.event).toBe("users.signin");
expect(parsedBody.payload.id).toBe(signedInUser.id);
expect(parsedBody.payload.model).toBeDefined();
const deliveries = await WebhookDelivery.findAll({
where: { webhookSubscriptionId: subscription.id },
});
expect(deliveries.length).toBe(1);
const delivery = deliveries[0];
expect(delivery.status).toBe("success");
expect(delivery.statusCode).toBe(200);
expect(delivery.responseBody).toEqual("SUCCESS");
});
test("should hit the subscription url when the eventing model doesn't exist", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
const deletedUserId = uuidv4();
const signedInUser = await buildUser({ teamId: subscription.teamId });
const task = new DeliverWebhookTask();
const event: UserEvent = {
name: "users.delete",
userId: deletedUserId,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await task.perform({
event,
subscriptionId: subscription.id,
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock).toHaveBeenCalledWith(
"http://example.com",
expect.anything()
);
const parsedBody = JSON.parse(
fetchMock.mock.calls[0]![1]!.body!.toString()
);
expect(parsedBody.webhookSubscriptionId).toBe(subscription.id);
expect(parsedBody.event).toBe("users.delete");
expect(parsedBody.payload.id).toBe(deletedUserId);
const deliveries = await WebhookDelivery.findAll({
where: { webhookSubscriptionId: subscription.id },
});
expect(deliveries.length).toBe(1);
const delivery = deliveries[0];
expect(delivery.status).toBe("success");
expect(delivery.statusCode).toBe(200);
expect(delivery.responseBody).toBeDefined();
});
test("should mark delivery as failed if post fails", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
fetchMock.mockResponse("FAILED", { status: 500 });
const signedInUser = await buildUser({ teamId: subscription.teamId });
const task = new DeliverWebhookTask();
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await task.perform({
event,
subscriptionId: subscription.id,
});
await subscription.reload();
expect(subscription.enabled).toBe(true);
const deliveries = await WebhookDelivery.findAll({
where: { webhookSubscriptionId: subscription.id },
});
expect(deliveries.length).toBe(1);
const delivery = deliveries[0];
expect(delivery.status).toBe("failed");
expect(delivery.statusCode).toBe(500);
expect(delivery.responseBody).toBeDefined();
expect(delivery.responseBody).toEqual("FAILED");
});
test("should disable the subscription if past deliveries failed", async () => {
const subscription = await buildWebhookSubscription({
url: "http://example.com",
events: ["*"],
});
for (let i = 0; i < 25; i++) {
await buildWebhookDelivery({
webhookSubscriptionId: subscription.id,
status: "failed",
});
}
fetchMock.mockResponse(JSON.stringify({ message: "Failure" }), {
status: 500,
});
const signedInUser = await buildUser({ teamId: subscription.teamId });
const task = new DeliverWebhookTask();
const event: UserEvent = {
name: "users.signin",
userId: signedInUser.id,
teamId: subscription.teamId,
actorId: signedInUser.id,
ip,
};
await task.perform({
event,
subscriptionId: subscription.id,
});
await subscription.reload();
expect(subscription.enabled).toBe(false);
const deliveries = await WebhookDelivery.findAll({
where: { webhookSubscriptionId: subscription.id },
order: [["createdAt", "DESC"]],
});
expect(deliveries.length).toBe(26);
const delivery = deliveries[0];
expect(delivery.status).toBe("failed");
expect(delivery.statusCode).toBe(500);
expect(delivery.responseBody).toEqual('{"message":"Failure"}');
});
});

560
server/queues/tasks/DeliverWebhookTask.ts Normal file
View File

@@ -0,0 +1,560 @@
import invariant from "invariant";
import env from "@server/env";
import Logger from "@server/logging/Logger";
import {
Collection,
FileOperation,
Group,
Integration,
Pin,
Star,
Team,
WebhookDelivery,
WebhookSubscription,
Document,
User,
Revision,
View,
Share,
CollectionUser,
CollectionGroup,
GroupUser,
} from "@server/models";
import {
presentCollection,
presentDocument,
presentRevision,
presentFileOperation,
presentGroup,
presentIntegration,
presentPin,
presentStar,
presentTeam,
presentUser,
presentWebhook,
presentWebhookSubscription,
presentView,
presentShare,
presentMembership,
presentGroupMembership,
presentCollectionGroupMembership,
} from "@server/presenters";
import { WebhookPayload } from "@server/presenters/webhook";
import {
CollectionEvent,
CollectionGroupEvent,
CollectionUserEvent,
DocumentEvent,
Event,
FileOperationEvent,
GroupEvent,
GroupUserEvent,
IntegrationEvent,
PinEvent,
RevisionEvent,
ShareEvent,
StarEvent,
TeamEvent,
UserEvent,
ViewEvent,
WebhookSubscriptionEvent,
} from "@server/types";
import BaseTask from "./BaseTask";
function assertUnreachable(event: never) {
Logger.warn(`DeliverWebhookTask did not handle ${(event as any).name}`);
}
type Props = {
subscriptionId: string;
event: Event;
};
export default class DeliverWebhookTask extends BaseTask<Props> {
public async perform({ subscriptionId, event }: Props) {
const subscription = await WebhookSubscription.findByPk(subscriptionId);
invariant(subscription, "Subscription not found");
Logger.info(
"task",
`DeliverWebhookTask: ${event.name} for ${subscription.name}`
);
switch (event.name) {
case "api_keys.create":
case "api_keys.delete":
// Ignored
return;
case "users.create":
case "users.signin":
case "users.signout":
case "users.update":
case "users.suspend":
case "users.activate":
case "users.delete":
case "users.invite":
await this.handleUserEvent(subscription, event);
return;
case "documents.create":
case "documents.publish":
case "documents.unpublish":
case "documents.delete":
case "documents.permanent_delete":
case "documents.archive":
case "documents.unarchive":
case "documents.restore":
case "documents.star":
case "documents.unstar":
case "documents.move":
case "documents.update":
case "documents.title_change":
await this.handleDocumentEvent(subscription, event);
return;
case "documents.update.delayed":
case "documents.update.debounced":
// Ignored
return;
case "revisions.create":
await this.handleRevisionEvent(subscription, event);
return;
case "fileOperations.create":
case "fileOperations.update":
case "fileOperation.delete":
await this.handleFileOperationEvent(subscription, event);
return;
case "collections.create":
case "collections.update":
case "collections.delete":
case "collections.move":
case "collections.permission_changed":
await this.handleCollectionEvent(subscription, event);
return;
case "collections.add_user":
case "collections.remove_user":
await this.handleCollectionUserEvent(subscription, event);
return;
case "collections.add_group":
case "collections.remove_group":
await this.handleCollectionGroupEvent(subscription, event);
return;
case "groups.create":
case "groups.update":
case "groups.delete":
await this.handleGroupEvent(subscription, event);
return;
case "groups.add_user":
case "groups.remove_user":
await this.handleGroupUserEvent(subscription, event);
return;
case "integrations.create":
case "integrations.update":
await this.handleIntegrationEvent(subscription, event);
return;
case "teams.update":
await this.handleTeamEvent(subscription, event);
return;
case "pins.create":
case "pins.update":
case "pins.delete":
await this.handlePinEvent(subscription, event);
return;
case "stars.create":
case "stars.update":
case "stars.delete":
await this.handleStarEvent(subscription, event);
return;
case "shares.create":
case "shares.update":
case "shares.revoke":
await this.handleShareEvent(subscription, event);
return;
case "webhook_subscriptions.create":
case "webhook_subscriptions.delete":
case "webhook_subscriptions.update":
await this.handleWebhookSubscriptionEvent(subscription, event);
return;
case "views.create":
await this.handleViewEvent(subscription, event);
return;
default:
assertUnreachable(event);
}
}
private async handleWebhookSubscriptionEvent(
subscription: WebhookSubscription,
event: WebhookSubscriptionEvent
): Promise<void> {
const model = await WebhookSubscription.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentWebhookSubscription(model),
},
});
}
private async handleViewEvent(
subscription: WebhookSubscription,
event: ViewEvent
): Promise<void> {
const model = await View.scope("withUser").findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentView(model),
},
});
}
private async handleStarEvent(
subscription: WebhookSubscription,
event: StarEvent
): Promise<void> {
const model = await Star.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentStar(model),
},
});
}
private async handleShareEvent(
subscription: WebhookSubscription,
event: ShareEvent
): Promise<void> {
const model = await Share.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentShare(model),
},
});
}
private async handlePinEvent(
subscription: WebhookSubscription,
event: PinEvent
): Promise<void> {
const model = await Pin.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentPin(model),
},
});
}
private async handleTeamEvent(
subscription: WebhookSubscription,
event: TeamEvent
): Promise<void> {
const model = await Team.scope("withDomains").findByPk(event.teamId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.teamId,
model: model && presentTeam(model),
},
});
}
private async handleIntegrationEvent(
subscription: WebhookSubscription,
event: IntegrationEvent
): Promise<void> {
const model = await Integration.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentIntegration(model),
},
});
}
private async handleGroupEvent(
subscription: WebhookSubscription,
event: GroupEvent
): Promise<void> {
const model = await Group.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentGroup(model),
},
});
}
private async handleGroupUserEvent(
subscription: WebhookSubscription,
event: GroupUserEvent
): Promise<void> {
const model = await GroupUser.scope(["withUser", "withGroup"]).findOne({
where: {
groupId: event.modelId,
userId: event.userId,
},
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: `${event.userId}-${event.modelId}`,
model: model && presentGroupMembership(model),
group: model && presentGroup(model.group),
user: model && presentUser(model.user),
},
});
}
private async handleCollectionEvent(
subscription: WebhookSubscription,
event: CollectionEvent
): Promise<void> {
const model = await Collection.findByPk(event.collectionId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.collectionId,
model: model && presentCollection(model),
},
});
}
private async handleCollectionUserEvent(
subscription: WebhookSubscription,
event: CollectionUserEvent
): Promise<void> {
const model = await CollectionUser.scope([
"withUser",
"withCollection",
]).findOne({
where: {
collectionId: event.collectionId,
userId: event.userId,
},
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: `${event.userId}-${event.collectionId}`,
model: model && presentMembership(model),
collection: model && presentCollection(model.collection),
user: model && presentUser(model.user),
},
});
}
private async handleCollectionGroupEvent(
subscription: WebhookSubscription,
event: CollectionGroupEvent
): Promise<void> {
const model = await CollectionGroup.scope([
"withGroup",
"withCollection",
]).findOne({
where: {
collectionId: event.collectionId,
groupId: event.modelId,
},
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: `${event.modelId}-${event.collectionId}`,
model: model && presentCollectionGroupMembership(model),
collection: model && presentCollection(model.collection),
group: model && presentGroup(model.group),
},
});
}
private async handleFileOperationEvent(
subscription: WebhookSubscription,
event: FileOperationEvent
): Promise<void> {
const model = await FileOperation.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && presentFileOperation(model),
},
});
}
private async handleDocumentEvent(
subscription: WebhookSubscription,
event: DocumentEvent
): Promise<void> {
const model = await Document.findByPk(event.documentId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.documentId,
model: model && (await presentDocument(model)),
},
});
}
private async handleRevisionEvent(
subscription: WebhookSubscription,
event: RevisionEvent
): Promise<void> {
const model = await Revision.findByPk(event.modelId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.modelId,
model: model && (await presentRevision(model)),
},
});
}
private async handleUserEvent(
subscription: WebhookSubscription,
event: UserEvent
): Promise<void> {
const model = await User.findByPk(event.userId, {
paranoid: false,
});
await this.sendWebhook({
event,
subscription,
payload: {
id: event.userId,
model: model && presentUser(model),
},
});
}
private async sendWebhook({
event,
subscription,
payload,
}: {
event: Event;
subscription: WebhookSubscription;
payload: WebhookPayload;
}) {
const delivery = await WebhookDelivery.create({
webhookSubscriptionId: subscription.id,
status: "pending",
});
let response, requestBody, requestHeaders, status;
try {
requestBody = presentWebhook({
event,
delivery,
payload,
});
requestHeaders = {
"Content-Type": "application/json",
"user-agent": `Outline-Webhooks${env.VERSION ? `/${env.VERSION}` : ""}`,
};
response = await fetch(subscription.url, {
method: "POST",
headers: requestHeaders,
body: JSON.stringify(requestBody),
});
status = response.ok ? "success" : "failed";
} catch (err) {
status = "failed";
}
await delivery.update({
status,
statusCode: response ? response.status : null,
requestBody,
requestHeaders,
responseBody: response ? await response.text() : "",
responseHeaders: response
? Object.fromEntries(response.headers.entries())
: {},
});
if (response && !response.ok) {
const recentDeliveries = await WebhookDelivery.findAll({
where: {
webhookSubscriptionId: subscription.id,
},
order: [["createdAt", "DESC"]],
limit: 25,
});
const allFailed = recentDeliveries.every(
(delivery) => delivery.status === "failed"
);
if (recentDeliveries.length === 25 && allFailed) {
await subscription.update({ enabled: false });
}
}
}
}

6
server/routes/api/auth.ts
View File

@@ -5,7 +5,7 @@ import { parseDomain } from "@shared/utils/domains";
import { sequelize } from "@server/database/sequelize";
import env from "@server/env";
import auth from "@server/middlewares/authentication";
import { Event, Team, TeamDomain } from "@server/models";
import { Event, Team } from "@server/models";
import { presentUser, presentTeam, presentPolicies } from "@server/presenters";
import ValidateSSOAccessTask from "@server/queues/tasks/ValidateSSOAccessTask";
import providers from "../auth/providers";
@@ -108,9 +108,7 @@ router.post("auth.config", async (ctx) => {
router.post("auth.info", auth(), async (ctx) => {
const { user } = ctx.state;
const team = await Team.findByPk(user.teamId, {
include: [{ model: TeamDomain }],
});
const team = await Team.scope("withDomains").findByPk(user.teamId);
invariant(team, "Team not found");
await ValidateSSOAccessTask.schedule({ userId: user.id });

3
server/routes/api/cron.ts
View File

@@ -5,6 +5,7 @@ import { AuthenticationError } from "@server/errors";
import CleanupDeletedDocumentsTask from "@server/queues/tasks/CleanupDeletedDocumentsTask";
import CleanupDeletedTeamsTask from "@server/queues/tasks/CleanupDeletedTeamsTask";
import CleanupExpiredFileOperationsTask from "@server/queues/tasks/CleanupExpiredFileOperationsTask";
import CleanupWebhookDeliveriesTask from "@server/queues/tasks/CleanupWebhookDeliveriesTask";
import InviteReminderTask from "@server/queues/tasks/InviteReminderTask";
const router = new Router();
@@ -22,6 +23,8 @@ const cronHandler = async (ctx: Context) => {
await CleanupDeletedTeamsTask.schedule({ limit });
await CleanupWebhookDeliveriesTask.schedule({ limit });
await InviteReminderTask.schedule();
ctx.body = {

12
server/routes/api/groups.ts
View File

@@ -45,7 +45,9 @@ router.post("groups.list", auth(), pagination(), async (ctx) => {
.slice(0, MAX_AVATAR_DISPLAY)
)
.flat()
.map(presentGroupMembership),
.map((membership) =>
presentGroupMembership(membership, { includeUser: true })
),
},
policies: presentPolicies(user, groups),
};
@@ -191,7 +193,9 @@ router.post("groups.memberships", auth(), pagination(), async (ctx) => {
ctx.body = {
pagination: ctx.state.pagination,
data: {
groupMemberships: memberships.map(presentGroupMembership),
groupMemberships: memberships.map((membership) =>
presentGroupMembership(membership, { includeUser: true })
),
users: memberships.map((membership) => presentUser(membership.user)),
},
};
@@ -250,7 +254,9 @@ router.post("groups.add_user", auth(), async (ctx) => {
ctx.body = {
data: {
users: [presentUser(user)],
groupMemberships: [presentGroupMembership(membership)],
groupMemberships: [
presentGroupMembership(membership, { includeUser: true }),
],
groups: [presentGroup(group)],
},
};

2
server/routes/api/index.ts
View File

@@ -27,6 +27,7 @@ import stars from "./stars";
import team from "./team";
import users from "./users";
import views from "./views";
import webhookSubscriptions from "./webhookSubscriptions";
const api = new Koa();
const router = new Router();
@@ -67,6 +68,7 @@ router.use("/", attachments.routes());
router.use("/", utils.routes());
router.use("/", groups.routes());
router.use("/", fileOperationsRoute.routes());
router.use("/", webhookSubscriptions.routes());
router.post("*", (ctx) => {
ctx.throw(NotFoundError("Endpoint not found"));

1
server/routes/api/views.ts
View File

@@ -44,6 +44,7 @@ router.post("views.create", auth(), async (ctx) => {
documentId: document.id,
collectionId: document.collectionId,
teamId: user.teamId,
modelId: view.id,
data: {
title: document.title,
},

137
server/routes/api/webhookSubscriptions.ts Normal file
View File

@@ -0,0 +1,137 @@
import Router from "koa-router";
import { compact } from "lodash";
import { ValidationError } from "@server/errors";
import auth from "@server/middlewares/authentication";
import { WebhookSubscription, Event } from "@server/models";
import { authorize } from "@server/policies";
import { presentWebhookSubscription } from "@server/presenters";
import { WebhookSubscriptionEvent } from "@server/types";
import { assertArray, assertPresent, assertUuid } from "@server/validation";
import pagination from "./middlewares/pagination";
const router = new Router();
router.post("webhookSubscriptions.list", auth(), pagination(), async (ctx) => {
const { user } = ctx.state;
authorize(user, "listWebhookSubscription", user.team);
const webhooks = await WebhookSubscription.findAll({
where: {
teamId: user.teamId,
},
order: [["createdAt", "DESC"]],
offset: ctx.state.pagination.offset,
limit: ctx.state.pagination.limit,
});
ctx.body = {
pagination: ctx.state.pagination,
data: webhooks.map(presentWebhookSubscription),
};
});
router.post("webhookSubscriptions.create", auth(), async (ctx) => {
const { user } = ctx.state;
authorize(user, "createWebhookSubscription", user.team);
const { name, url } = ctx.request.body;
const events: string[] = compact(ctx.request.body.events);
assertPresent(name, "name is required");
assertPresent(url, "url is required");
assertArray(events, "events is required");
if (events.length === 0) {
throw ValidationError("events are required");
}
const webhookSubscription = await WebhookSubscription.create({
name,
events,
createdById: user.id,
teamId: user.teamId,
url,
enabled: true,
});
const event: WebhookSubscriptionEvent = {
name: "webhook_subscriptions.create",
modelId: webhookSubscription.id,
teamId: user.teamId,
actorId: user.id,
data: {
name,
url,
events,
},
ip: ctx.request.ip,
};
await Event.create(event);
ctx.body = {
data: presentWebhookSubscription(webhookSubscription),
};
});
router.post("webhookSubscriptions.delete", auth(), async (ctx) => {
const { id } = ctx.body;
assertUuid(id, "id is required");
const { user } = ctx.state;
const webhookSubscription = await WebhookSubscription.findByPk(id);
authorize(user, "delete", webhookSubscription);
await webhookSubscription.destroy();
const event: WebhookSubscriptionEvent = {
name: "webhook_subscriptions.delete",
modelId: webhookSubscription.id,
teamId: user.teamId,
actorId: user.id,
data: {
name: webhookSubscription.name,
url: webhookSubscription.url,
events: webhookSubscription.events,
},
ip: ctx.request.ip,
};
await Event.create(event);
});
router.post("webhookSubscriptions.update", auth(), async (ctx) => {
const { id } = ctx.body;
assertUuid(id, "id is required");
const { user } = ctx.state;
const { name, url } = ctx.request.body;
const events: string[] = compact(ctx.request.body.events);
assertPresent(name, "name is required");
assertPresent(url, "url is required");
assertArray(events, "events is required");
if (events.length === 0) {
throw ValidationError("events are required");
}
const webhookSubscription = await WebhookSubscription.findByPk(id);
authorize(user, "update", webhookSubscription);
await webhookSubscription.update({ name, url, events, enabled: true });
const event: WebhookSubscriptionEvent = {
name: "webhook_subscriptions.update",
modelId: webhookSubscription.id,
teamId: user.teamId,
actorId: user.id,
data: {
name: webhookSubscription.name,
url: webhookSubscription.url,
events: webhookSubscription.events,
},
ip: ctx.request.ip,
};
await Event.create(event);
ctx.body = {
data: presentWebhookSubscription(webhookSubscription),
};
});
export default router;

57
server/test/factories.ts
View File

@@ -14,6 +14,8 @@ import {
Integration,
AuthenticationProvider,
FileOperation,
WebhookSubscription,
WebhookDelivery,
} from "@server/models";
import {
FileOperationState,
@@ -366,3 +368,58 @@ export async function buildAttachment(overrides: Partial<Attachment> = {}) {
...overrides,
});
}
export async function buildWebhookSubscription(
overrides: Partial<WebhookSubscription> = {}
): Promise<WebhookSubscription> {
if (!overrides.teamId) {
const team = await buildTeam();
overrides.teamId = team.id;
}
if (!overrides.createdById) {
const user = await buildUser({
teamId: overrides.teamId,
});
overrides.createdById = user.id;
}
if (!overrides.name) {
overrides.name = "Test Webhook Subscription";
}
if (!overrides.url) {
overrides.url = "https://www.example.com/webhook";
}
if (!overrides.events) {
overrides.events = ["*"];
}
if (!overrides.enabled) {
overrides.enabled = true;
}
return WebhookSubscription.create(overrides);
}
export async function buildWebhookDelivery(
overrides: Partial<WebhookDelivery> = {}
): Promise<WebhookDelivery> {
if (!overrides.status) {
overrides.status = "success";
}
if (!overrides.statusCode) {
overrides.statusCode = 200;
}
if (!overrides.requestBody) {
overrides.requestBody = "{}";
}
if (!overrides.requestHeaders) {
overrides.requestHeaders = {};
}
if (!overrides.webhookSubscriptionId) {
const webhookSubscription = await buildWebhookSubscription();
overrides.webhookSubscriptionId = webhookSubscription.id;
}
if (!overrides.createdAt) {
overrides.createdAt = new Date();
}
return WebhookDelivery.create(overrides);
}

389
server/types.ts
View File

@@ -1,5 +1,5 @@
import { Context } from "koa";
import { FileOperation, User } from "./models";
import { FileOperation, Team, User } from "./models";
export type ContextWithState = Context & {
state: {
@@ -9,227 +9,241 @@ export type ContextWithState = Context & {
};
};
export type UserEvent =
| {
name: "users.create" // eslint-disable-line
| "users.signin"
| "users.signout"
| "users.update"
| "users.suspend"
| "users.activate"
| "users.delete";
userId: string;
teamId: string;
actorId: string;
ip: string;
}
| {
name: "users.invite";
teamId: string;
actorId: string;
data: {
email: string;
name: string;
};
ip: string;
};
type BaseEvent = {
teamId: string;
actorId: string;
ip: string;
};
export type DocumentEvent =
| {
name: "documents.create" // eslint-disable-line
| "documents.publish"
| "documents.unpublish"
| "documents.delete"
| "documents.permanent_delete"
| "documents.archive"
| "documents.unarchive"
| "documents.restore"
| "documents.star"
| "documents.unstar";
documentId: string;
collectionId: string;
teamId: string;
actorId: string;
ip: string;
data: {
title: string;
source?: "import";
};
}
| {
name: "documents.move";
documentId: string;
collectionId: string;
teamId: string;
actorId: string;
data: {
collectionIds: string[];
documentIds: string[];
};
ip: string;
}
| {
name: "documents.update" // eslint-disable-line
| "documents.update.delayed"
| "documents.update.debounced";
documentId: string;
collectionId: string;
createdAt: string;
teamId: string;
actorId: string;
data: {
title: string;
autosave: boolean;
done: boolean;
};
ip: string;
}
| {
name: "documents.title_change";
documentId: string;
collectionId: string;
createdAt: string;
teamId: string;
actorId: string;
data: {
title: string;
previousTitle: string;
};
ip: string;
};
export type ApiKeyEvent = BaseEvent & {
name: "api_keys.create" | "api_keys.delete";
modelId: string;
data: {
name: string;
};
};
export type RevisionEvent = {
export type UserEvent = BaseEvent &
(
| {
name:
| "users.create"
| "users.signin"
| "users.signout"
| "users.update"
| "users.suspend"
| "users.activate"
| "users.delete";
userId: string;
}
| {
name: "users.invite";
userId: string;
data: {
email: string;
name: string;
};
}
);
export type DocumentEvent = BaseEvent &
(
| {
name:
| "documents.create"
| "documents.publish"
| "documents.unpublish"
| "documents.delete"
| "documents.permanent_delete"
| "documents.archive"
| "documents.unarchive"
| "documents.restore"
| "documents.star"
| "documents.unstar";
documentId: string;
collectionId: string;
data: {
title: string;
source?: "import";
};
}
| {
name: "documents.move";
documentId: string;
collectionId: string;
data: {
collectionIds: string[];
documentIds: string[];
};
}
| {
name:
| "documents.update"
| "documents.update.delayed"
| "documents.update.debounced";
documentId: string;
collectionId: string;
createdAt: string;
data: {
title: string;
autosave: boolean;
done: boolean;
};
}
| {
name: "documents.title_change";
documentId: string;
collectionId: string;
createdAt: string;
data: {
title: string;
previousTitle: string;
};
}
);
export type RevisionEvent = BaseEvent & {
name: "revisions.create";
documentId: string;
collectionId: string;
teamId: string;
modelId: string;
};
export type FileOperationEvent = {
export type FileOperationEvent = BaseEvent & {
name:
| "fileOperations.create"
| "fileOperations.update"
| "fileOperation.delete";
teamId: string;
actorId: string;
modelId: string;
data: Partial<FileOperation>;
};
export type CollectionEvent =
| {
name: "collections.create" // eslint-disable-line
| "collections.update"
| "collections.delete";
collectionId: string;
teamId: string;
actorId: string;
data: {
name: string;
};
ip: string;
}
| {
name: "collections.add_user" | "collections.remove_user";
userId: string;
collectionId: string;
teamId: string;
actorId: string;
ip: string;
}
| {
name: "collections.add_group" | "collections.remove_group";
collectionId: string;
teamId: string;
actorId: string;
modelId: string;
data: {
name: string;
};
ip: string;
}
| {
name: "collections.move";
collectionId: string;
teamId: string;
actorId: string;
data: {
index: string;
};
ip: string;
}
| {
name: "collections.permission_changed";
collectionId: string;
teamId: string;
actorId: string;
data: {
privacyChanged: boolean;
sharingChanged: boolean;
};
ip: string;
};
export type CollectionUserEvent = BaseEvent & {
name: "collections.add_user" | "collections.remove_user";
userId: string;
collectionId: string;
};
export type GroupEvent =
| {
name: "groups.create" | "groups.delete" | "groups.update";
actorId: string;
modelId: string;
teamId: string;
data: {
name: string;
};
ip: string;
}
| {
name: "groups.add_user" | "groups.remove_user";
actorId: string;
userId: string;
modelId: string;
teamId: string;
data: {
name: string;
};
ip: string;
};
export type CollectionGroupEvent = BaseEvent & {
name: "collections.add_group" | "collections.remove_group";
collectionId: string;
modelId: string;
data: {
name: string;
};
};
export type IntegrationEvent = {
export type CollectionEvent = BaseEvent &
(
| CollectionUserEvent
| CollectionGroupEvent
| {
name:
| "collections.create"
| "collections.update"
| "collections.delete";
collectionId: string;
data: {
name: string;
};
}
| {
name: "collections.move";
collectionId: string;
data: {
index: string;
};
}
| {
name: "collections.permission_changed";
collectionId: string;
data: {
privacyChanged: boolean;
sharingChanged: boolean;
};
}
);
export type GroupUserEvent = BaseEvent & {
name: "groups.add_user" | "groups.remove_user";
userId: string;
modelId: string;
data: {
name: string;
};
};
export type GroupEvent = BaseEvent &
(
| GroupUserEvent
| {
name: "groups.create" | "groups.delete" | "groups.update";
modelId: string;
data: {
name: string;
};
}
);
export type IntegrationEvent = BaseEvent & {
name: "integrations.create" | "integrations.update";
modelId: string;
teamId: string;
actorId: string;
ip: string;
};
export type TeamEvent = {
export type TeamEvent = BaseEvent & {
name: "teams.update";
teamId: string;
actorId: string;
data: Record<string, any>;
ip: string;
data: Partial<Team>;
};
export type PinEvent = {
export type PinEvent = BaseEvent & {
name: "pins.create" | "pins.update" | "pins.delete";
teamId: string;
modelId: string;
documentId: string;
collectionId?: string;
actorId: string;
ip: string;
};
export type StarEvent = {
export type StarEvent = BaseEvent & {
name: "stars.create" | "stars.update" | "stars.delete";
teamId: string;
modelId: string;
documentId: string;
userId: string;
actorId: string;
ip: string;
};
export type ShareEvent = BaseEvent & {
name: "shares.create" | "shares.update" | "shares.revoke";
modelId: string;
documentId: string;
collectionId?: string;
data: {
name: string;
};
};
export type ViewEvent = BaseEvent & {
name: "views.create";
documentId: string;
collectionId: string;
modelId: string;
data: {
title: string;
};
};
export type WebhookSubscriptionEvent = BaseEvent & {
name:
| "webhook_subscriptions.create"
| "webhook_subscriptions.delete"
| "webhook_subscriptions.update";
modelId: string;
data: {
name: string;
url: string;
events: string[];
};
};
export type Event =
| ApiKeyEvent
| UserEvent
| DocumentEvent
| PinEvent
@@ -239,4 +253,7 @@ export type Event =
| IntegrationEvent
| GroupEvent
| RevisionEvent
| TeamEvent;
| ShareEvent
| TeamEvent
| ViewEvent
| WebhookSubscriptionEvent;