Commit Graph

369 Commits

Author SHA1 Message Date
Tom Moor
b8115ae3ce fix: Add url validation to team and user avatar fields 2022-08-30 23:05:57 +02:00
Apoorv Mishra
4dbad4e46c feat: Support embed configuration (#3980)
* wip

* stash

* fix: make authenticationId nullable fk

* fix: apply generics to resolve compile time type errors

* fix: loosen integration settings

* chore: refactor into functional component

* feat: pass integrations all the way to embeds

* perf: avoid re-fetching integrations

* fix: change attr name to avoid type overlap

* feat: use hostname from embed settings in matcher

* Revert "feat: use hostname from embed settings in matcher"

This reverts commit e7485d9cda4dcf45104e460465ca104a56c67ddc.

* feat: refactor  into a class

* chore: refactor url regex formation as a util

* fix: escape regex special chars

* fix: remove in-house escapeRegExp in favor of lodash's

* fix: sanitize url

* perf: memoize embeds

* fix: rename hostname to url and allow spreading entire settings instead of just url

* fix: replace diagrams with drawio

* fix: rename

* fix: support self-hosted and saas both

* fix: assert on settings url

* fix: move embed integrations loading to hook

* fix: address review comments

* fix: use observer in favor of explicit state setters

* fix: refactor useEmbedIntegrations into useEmbeds

* fix: use translations for toasts

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-26 12:21:46 +05:30
CuriousCorrelation
24c71c38a5 feat: Document subscriptions (#3834)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-25 23:47:13 -07:00
Tom Moor
864f585e5b chore: Remove long deprecated database columns (#3821)
* chore: Remove long deprecated database columns

* test

* Update 20220720221531-remove-deprecated-columns.js

* fix rollback

* Add guard for upgrading past v0.54.0
2022-08-25 11:52:01 -07:00
Tom Moor
f32f07cdcc chore: Refactor user activation to command 2022-08-18 11:24:27 +02:00
Tom Moor
e57941732a fix: emoji column no longer filled in db, simplified state length validation 2022-08-16 22:05:10 +02:00
Tom Moor
cfa7ecd7f8 fix: Add missing validation to document state 2022-08-16 09:35:31 +02:00
Tom Moor
a326e0ee88 chore: Rate limiter audit (#3965)
* chore: Rate limiter audit api/users

* Make requests required

* api/collections

* Remove checkRateLimit on FileOperation (now done at route level through rate limiter)

* auth rate limit

* Add metric logging when rate limit exceeded

* Refactor to shared configs

* test
2022-08-14 08:04:04 -07:00
Apoorv Mishra
0c51bfb899 perf: reduce memory usage upon running server tests (#3949)
* perf: reduce memory usage upon running server tests

* perf: plug leaks in server/routes

* perf: plug leaks in server/scripts

* perf: plug leaks in server/policies

* perf: plug leaks in server/models

* perf: plug leaks in server/middlewares

* perf: plug leaks in server/commands

* fix: missing await on db.flush

* perf: plug leaks in server/queues

* chore: remove unused legacy funcs

* fix: await on db.flush

* perf: await on GC to run in between tests

* fix: remove db refs

* fix: revert embeds

* perf: plug leaks in shared/i18n
2022-08-11 21:39:17 +05:30
akp
8e1f42a9cb Add optional export notifications (#3935)
* Add `emails.export_completed` notification to settings menu

Signed-off-by: AKP <tom@tdpain.net>

* Don't send email when export_completed notifications are disabled

Signed-off-by: AKP <tom@tdpain.net>

* Automatically subscribe new users to `export_completed` notifications

Signed-off-by: AKP <tom@tdpain.net>

* Alter secondary text on export page to mention optional notifications

Signed-off-by: AKP <tom@tdpain.net>

* Alter toast text on collection export for optional notifications

Signed-off-by: AKP <tom@tdpain.net>

* Only subscribe new admins to export notifs

Signed-off-by: AKP <tom@tdpain.net>

* Move `export_completed` notification decision into `beforeSend`

Signed-off-by: AKP <tom@tdpain.net>

* Update server/emails/templates/ExportFailureEmail.tsx

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update server/emails/templates/ExportSuccessEmail.tsx

Co-authored-by: Tom Moor <tom.moor@gmail.com>

Signed-off-by: AKP <tom@tdpain.net>
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-11 07:31:35 -07:00
Tom Moor
1d242d44b1 chore: Add eslint rule for object shorthand (#3955) 2022-08-11 05:18:14 -07:00
Tom Moor
b45e6c504f fix: Prevent webhook delivery for deleted teams 2022-08-08 11:15:04 +02:00
Tom Moor
b37a848914 Add limit of 10 webhooks/team 2022-08-08 10:58:47 +02:00
Tom Moor
cb9773ad85 chore: Add emailed confirmation code to account deletion (#3873)
* wip

* tests
2022-07-31 10:59:40 -07:00
Tom Moor
ec35af4bc5 Refactor validations 2022-07-24 13:40:04 +01:00
Apoorv Mishra
24170e8684 chore: Remove updatedAt column from events table (#3841) 2022-07-24 01:57:21 -07:00
Tom Moor
7ae892fe06 fix: Long collection description prevents import (#3847)
* fix: Long collection description prevents import
fix: Parallelize attachment upload during import

* fix: Improve Notion image import matching

* chore: Bump JSZIP (perf)

* fix: Allow redirect from /doc/<id> to canonical url

* fix: Importing document with only title duplicates title in body
2022-07-24 01:37:20 -07:00
Tom Moor
4bca081faa chore: Add rolling window limits to import and export operations 2022-07-23 16:29:28 +01:00
Tom Moor
47e73cee4e feat: Cleanup api keys and webhooks for suspended users (#3756) 2022-07-13 00:59:31 -07:00
Tom Moor
8bb88b8550 chore: Audit of all model column validations (#3757)
* chore: Updating all model validations before the white-hatters get to it ;)

* test

* Remove isUrl validation, thinking about it need to account for minio and other weird urls here
2022-07-09 08:04:40 -07:00
Tom Moor
4a46d19846 fix: Improved model validation for Collection (#3749)
* fix: Added improved model validation for Collection attributes

* sp

* fix: Enforce title length in UI
2022-07-08 11:10:22 -07:00
Tom Moor
ec8c0645ba fix: Correct annotation order 🙈 2022-07-07 12:23:27 +02:00
Tom Moor
f90309e781 fix: Unneccessary restrictive avatarUrl length 2022-07-07 12:16:54 +02:00
Tom Moor
c36e7bfbb6 fix: Add 10 domain limit per team (#3733)
* fix: Validate team domains are FQDN's
Add 10 domain limit per team
fix: Deletion of domains not happening within request lifecycle

* tests

* docs
2022-07-05 12:27:02 -07:00
Tom Moor
831df67358 feat: Adds route-level role filtering. (#3734)
* feat: Adds route-level role filtering. Another layer in the onion of security and performance

* fix: Regression in authentication middleware
2022-07-05 12:26:49 -07:00
Tom Moor
ee10e1407a fix: Typo of fileOperation -> fileOperations 2022-07-03 16:27:03 +02:00
Tom Moor
27978b8fc4 fix: Remove teams.create from audit events 2022-07-03 14:16:49 +02:00
Tom Moor
62d9bf7105 chore: Move initial avatar upload to background worker (#3727)
* chore: Async user avatar upload processor

* chore: Async team avatar upload

* Refactor to task for retries

* Docs
Include avatarUrl in task props to prevent race condition
Remove transaction around upload fetch request
2022-07-03 02:36:15 -07:00
Tom Moor
8ebe4b27b1 fix: Add additional model validation (#3725) 2022-07-02 14:29:01 -07:00
Tom Moor
0c30d2bb34 fix: share.document can be null when document is deleted
closes #3724
2022-07-02 19:56:15 +02:00
Tom Moor
f744d488f6 chore: Soft delete webhooks (#3722) 2022-07-02 10:41:28 -07:00
Tom Moor
863f22750f feat: Add optional notification email when invite is accepted (#3718)
* feat: Add optional notification email when invite is accepted

* Refactor to use beforeSend
2022-07-02 05:40:40 -07:00
CuriousCorrelation
adb55fa965 feat: Custom Length decorator for UTF-8 chars len (#3709)
* feat: Custom Length decorator for UTF-8 chars len

* fix: Length decorator function return type
2022-07-01 13:21:09 -07:00
Tom Moor
10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00
Tom Moor
6d4da176d1 chore: Move provisionSubdomain from Team model to teamCreator command 2022-06-22 11:09:20 +02:00
Tom Moor
305de71e8b chore: Block all email providers from being added as team domains (#3678) 2022-06-21 01:29:43 -07:00
Tom Moor
4b27feff61 fix: Enable documents.update with collab editing (#3647)
* fix: Enable documents.update with collab editing

* jest cannot deal with ESM deps
2022-06-20 06:36:25 -07:00
Tom Moor
118e5da345 fix: Unpublished does not appear in document history
closes #3429
2022-06-09 21:16:37 +02:00
Tom Moor
aed8d7a649 fix: SSR meta data for nested shared documents (#3646) 2022-06-08 01:38:34 -07:00
Tom Moor
80ad6cfec8 fix: Expired refreshToken should invalidate session, not check SSO retry task 2022-06-08 08:55:58 +02:00
Tom Moor
56393f39b7 fix: Previously provisioned JWT's should be revoked on signout (#3639)
* feat: auth.delete endpoint

* test
2022-06-07 13:57:17 -07:00
Tom Moor
0de6650aa5 chore: Suppress unneccessary model warnings from Sequelize upgrade 2022-06-07 09:38:00 +02:00
Tom Moor
e5b524e4c2 chore: Upgrade sequelize dependency 2022-06-06 21:54:54 +02:00
Tom Moor
728790e38f feat: Validate Google, Azure, OIDC SSO access (#3590)
* chore: Store expiresAt on UserAuthentications. This represents the time that the accessToken is no longer valid and should be exchanged using the refreshToken

* feat: Check and expire Google SSO

* fix: Better handling of multiple auth methods
Added more docs

* fix: Retry access validation with network errors

* Small refactor, add Azure token validation support

* doc

* test

* lint

* OIDC refresh support

* CheckSSOAccessTask -> ValidateSSOAccessTask
Added lastValidatedAt column
Skip checks if validated within 5min
Some edge cases around encrypted columns
2022-06-05 13:18:51 -07:00
Tom Moor
ac07724f21 chore: Synchronizing refactor and small fixes from enterprise codebase (#3634)
* chore: Syncronizing refactor and small fixes from enterprise codebase

* fix
2022-06-05 00:59:41 -07:00
Nan Yu
41e425756d chore: refactor domain parsing to be more general (#3448)
* change the api of domain parsing to just parseDomain and getCookieDomain
* adds getBaseDomain as the method to get the domain after any official subdomains
2022-05-31 18:48:23 -07:00
Tom Moor
bf0ff6c823 chore: Casing of logger -> Logger as it's an instantiated class 2022-05-21 13:59:23 +01:00
Tom Moor
f2be756cf4 feat: Improved error for community edition when database columns cannot be decrypted 2022-05-21 13:25:55 +01:00
Tom Moor
3c002f82cc chore: Centralize env parsing, validation, defaults, and deprecation notices (#3487)
* chore: Centralize env parsing, defaults, deprecation

* wip

* test

* test

* tsc

* docs, more validation

* fix: Allow empty REDIS_URL (defaults to localhost)

* test

* fix: SLACK_MESSAGE_ACTIONS not bool

* fix: Add SMTP port validation
2022-05-19 08:05:11 -07:00
Corey Alexander
51001cfac1 feat: Migrate allowedDomains to a Team Level Settings (#3489)
Fixes #3412

Previously the only way to restrict the domains for a Team were with the ALLOWED_DOMAINS environment variable for self hosted instances.
This PR migrates this to be a database backed setting on the Team object. This is done through the creation of a TeamDomain model that is associated with the Team and contains the domain name

This settings is updated on the Security Tab. Here domains can be added or removed from the Team.

On the server side, we take the code paths that previously were using ALLOWED_DOMAINS and switched them to use the Team allowed domains instead
2022-05-17 20:26:29 -04:00