Commit Graph

130 Commits

Author SHA1 Message Date
Apoorv Mishra
7a590550c9 Sign webhook requests (#4156)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-09-24 14:19:26 -07:00
Tom Moor
f8912732b8 chore: Flag users with platform used 2022-09-18 17:53:55 -04:00
Apoorv Mishra
6502b108e3 Introduce account preferences to remember user's previous location (#4126) 2022-09-18 06:01:47 -07:00
Apoorv Mishra
ce410c4bf3 Support user and team preferences (#4081)
* feat: support user preferences

* feat: support team preferences

* fix: update snapshots

* feat: update last visited url by user

* fix: update snapshots

* fix: use path instead of complete url

* fix: do not expose preferences to other users with the exception of admin

* feat: support defaultDocumentStatus as a team preference

* feat: allow edit even when collaborative editing is enabled

* Revert "feat: allow edit even when collaborative editing is enabled"

This reverts commit a22a02a406d01eb418dab32249b8b846bf77c59b.

* Revert "feat: support defaultDocumentStatus as a team preference"

This reverts commit 4928cffe5c682952b1e469a3e50a1a34d05dcc58.

* fix: keep preference as a boolean
2022-09-14 16:07:39 +05:30
Tom Moor
607a795dd0 tsc 2022-09-14 11:04:38 +01:00
Tom Moor
e1e7f1b97d fix: Include the maximum document import size in the error message 2022-09-14 09:20:17 +01:00
Tom Moor
afc7fb5f1d fixes #4104 2022-09-11 15:27:19 +01:00
Tom Moor
c89567991b fix: Unsure filename when downloading an untitled document
fix: Unsure unique filename when downloading revision
2022-09-11 14:32:38 +01:00
Tom Moor
0fd576cdd5 feat: Updated collection header (#4101)
* Return total results from collection membership endpoints

* Display membership preview on collections

* fix permissions

* Revert unneccessary changes
2022-09-11 05:54:57 -07:00
Tom Moor
1f93399447 feat: Add availableTeams to auth.info endpoint (#3981)
* Index emails migration

* feat: Add available teams to auth.info endpoint

* test

* separate presenter

* Include data from sessions cookie, include likely logged in state

* test

* test: Add test for team only in session cookie

* Suggested query change in PR feedback
2022-09-10 06:58:38 -07:00
Tom Moor
876803362f fix: Server error when code is passed as null to users.delete, closes #4070 2022-09-09 22:10:32 +01:00
Tom Moor
fa75d5585f feat: Show diff when navigating revision history (#4069)
* tidy

* Add title to HTML export

* fix: Add compatability for documents without collab state

* Add HTML download option to UI

* docs

* fix nodes that required document to render

* Refactor to allow for styling of HTML export

* div>article for easier programatic content extraction

* Allow DocumentHelper to be used with Revisions

* Add revisions.diff endpoint, first version

* Allow arbitrary revisions to be compared

* test

* HTML driven revision viewer

* fix: Dark mode styles for document diffs

* Add revision restore button to header

* test

* Support RTL languages in revision history viewer

* fix: RTL support
Remove unneccessary API requests

* Prefetch revision data

* Animate history sidebar

* fix: Cannot toggle history from timestamp
fix: Animation on each revision click

* Clarify currently editing history item
2022-09-08 02:17:52 -07:00
Tom Moor
e8a6de3f18 feat: Add HTML export option (#4056)
* tidy

* Add title to HTML export

* fix: Add compatability for documents without collab state

* Add HTML download option to UI

* docs

* fix nodes that required document to render

* Refactor to allow for styling of HTML export

* div>article for easier programatic content extraction
2022-09-07 04:34:39 -07:00
Tom Moor
34b586724b fix: Cannot download export result, closes #4059 2022-09-05 10:22:17 +02:00
Tom Moor
c85f3bd7b4 fix: Remove ability to use GET for RPC API requests by default (#4042)
* fix: Remove ability to use GET for RPC API requests by default

* tsc
2022-09-02 01:05:40 -07:00
Tom Moor
32b76303e5 Add simple count of views to share links (#4036)
* Add simple count of views to share links

* Remove no longer applicable tests

* Avoid incrementing view count for known bots
2022-08-30 23:16:40 -07:00
Tom Moor
212985e18f feat: Allow viewers to be upgraded to editors on individual collections (#4023)
* Improve types

* More types, fix default permission for viewers added to collection

* fix change of default role for CollectionGroup

* Restore policy

* test

* tests
2022-08-30 23:12:27 -07:00
Tom Moor
b8115ae3ce fix: Add url validation to team and user avatar fields 2022-08-30 23:05:57 +02:00
Tom Moor
1c8fadbe02 Merge branch 'tom/socket-refactor' 2022-08-27 11:51:38 +02:00
Apoorv Mishra
4dbad4e46c feat: Support embed configuration (#3980)
* wip

* stash

* fix: make authenticationId nullable fk

* fix: apply generics to resolve compile time type errors

* fix: loosen integration settings

* chore: refactor into functional component

* feat: pass integrations all the way to embeds

* perf: avoid re-fetching integrations

* fix: change attr name to avoid type overlap

* feat: use hostname from embed settings in matcher

* Revert "feat: use hostname from embed settings in matcher"

This reverts commit e7485d9cda4dcf45104e460465ca104a56c67ddc.

* feat: refactor  into a class

* chore: refactor url regex formation as a util

* fix: escape regex special chars

* fix: remove in-house escapeRegExp in favor of lodash's

* fix: sanitize url

* perf: memoize embeds

* fix: rename hostname to url and allow spreading entire settings instead of just url

* fix: replace diagrams with drawio

* fix: rename

* fix: support self-hosted and saas both

* fix: assert on settings url

* fix: move embed integrations loading to hook

* fix: address review comments

* fix: use observer in favor of explicit state setters

* fix: refactor useEmbedIntegrations into useEmbeds

* fix: use translations for toasts

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-26 12:21:46 +05:30
CuriousCorrelation
24c71c38a5 feat: Document subscriptions (#3834)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-25 23:47:13 -07:00
Tom Moor
354a68a8b7 Remove long-deprecated documents.star/documents.unstar 2022-08-25 21:51:34 +02:00
Tom Moor
421312b845 Possible fix for #3986 2022-08-22 09:47:47 +02:00
Tom Moor
72b0e78788 fix: Validate uuid on attachments.create endpoint 2022-08-20 23:46:01 +02:00
Tom Moor
f32f07cdcc chore: Refactor user activation to command 2022-08-18 11:24:27 +02:00
Tom Moor
7113b5f604 fix: Restore user deletion through API, increase rate limit 2022-08-17 22:40:00 +02:00
Tom Moor
a326e0ee88 chore: Rate limiter audit (#3965)
* chore: Rate limiter audit api/users

* Make requests required

* api/collections

* Remove checkRateLimit on FileOperation (now done at route level through rate limiter)

* auth rate limit

* Add metric logging when rate limit exceeded

* Refactor to shared configs

* test
2022-08-14 08:04:04 -07:00
Apoorv Mishra
0c51bfb899 perf: reduce memory usage upon running server tests (#3949)
* perf: reduce memory usage upon running server tests

* perf: plug leaks in server/routes

* perf: plug leaks in server/scripts

* perf: plug leaks in server/policies

* perf: plug leaks in server/models

* perf: plug leaks in server/middlewares

* perf: plug leaks in server/commands

* fix: missing await on db.flush

* perf: plug leaks in server/queues

* chore: remove unused legacy funcs

* fix: await on db.flush

* perf: await on GC to run in between tests

* fix: remove db refs

* fix: revert embeds

* perf: plug leaks in shared/i18n
2022-08-11 21:39:17 +05:30
Apoorv Mishra
7eaa8eb961 feat: Put request rate limit at application server (#3857)
* feat: Put request rate limit at application server

This PR contains implementation for a blanket rate limiter at
application server level. Currently the allowed throughput is set high
only to be changed later as per the actual data gathered.

* Simplify implementation

1. Remove shutdown handler to purge rate limiter keys
2. Have separate keys for default and custom(route-based) rate limiters
3. Do not kill default rate limiter because it is not needed anymore due
   to (2) above

* Set 60s as default for rate limiting window

* Fix env types
2022-08-11 15:40:30 +05:30
Tom Moor
71c9fcf59b test: Avoid creation of new server/app instance for each route test 2022-08-08 12:06:54 +02:00
Tom Moor
7923a7e071 Enforce user invites/request on server 2022-08-08 11:02:37 +02:00
Apoorv Mishra
0a6cfe5a6a feat: Choose random color on collection creation (#3912)
Choose a random color from a shared color palette between backend
and frontend during collection creation.
2022-08-04 01:48:19 -07:00
Tom Moor
cb9773ad85 chore: Add emailed confirmation code to account deletion (#3873)
* wip

* tests
2022-07-31 10:59:40 -07:00
Tom Moor
8fdd5bf734 fix: substitution of content when sending an image to a profile (#3869)
* fix: Limit public uploads to basic image types

* test
2022-07-26 12:10:00 -07:00
Tom Moor
11e1ef455f chore: Improve UUID vaildation – prevent nonsense reaching db queries 2022-07-17 14:49:04 +01:00
Tom Moor
e509719c77 Add ability to quickly create test users in development (#3764) 2022-07-15 16:11:30 -07:00
Tom Moor
47e73cee4e feat: Cleanup api keys and webhooks for suspended users (#3756) 2022-07-13 00:59:31 -07:00
Tom Moor
973cfc3fa3 Do not show suspended users to non admins (#3776) 2022-07-13 00:59:06 -07:00
Tom Moor
da4a10e877 chore: Remove shares.info apiVersion 1 (#3758)
* chore: Remove shares.info apiVersion 1

* fix: Sporadic test failure
2022-07-09 04:28:56 -07:00
Tom Moor
2893924e9a fix: Must check length before passing to timingSafeEqual 2022-07-09 11:19:40 +02:00
Tom Moor
97f8c0813c fix: Use crypto.timingSafeEqual, closes #3740 2022-07-08 21:10:51 +02:00
Nan Yu
1e808fc52c Feat: add auth provider to users on sign in (#3739)
* feat: merge a new authentication method onto existing user records when emails match

* adds test for invite acceptance and auth provider creation

* addresses comments
- test existing user and invites in different test cases
- update lastActiveAt syncronously when an invite is accepted

* sort arrays in test to prevent nondeterministic test behaivior when doing array compare
2022-07-08 00:24:46 -07:00
Tom Moor
831df67358 feat: Adds route-level role filtering. (#3734)
* feat: Adds route-level role filtering. Another layer in the onion of security and performance

* fix: Regression in authentication middleware
2022-07-05 12:26:49 -07:00
Tom Moor
9cd26168e1 Separates policy for file operations 2022-07-03 18:19:56 +02:00
Tom Moor
c9cd424a8d chore: Remove over-usage of invariant (#3719) 2022-07-02 05:29:39 -07:00
CuriousCorrelation
adb55fa965 feat: Custom Length decorator for UTF-8 chars len (#3709)
* feat: Custom Length decorator for UTF-8 chars len

* fix: Length decorator function return type
2022-07-01 13:21:09 -07:00
Tom Moor
7ce57c9c83 fix: attachments events not recognised by DeliverWebhookTask 2022-07-01 18:40:32 +02:00
Tom Moor
10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00
Tom Moor
ac467b2936 fix: Return direct url to public attachments, closes #3686 2022-06-24 11:24:11 +02:00
Tom Moor
305de71e8b chore: Block all email providers from being added as team domains (#3678) 2022-06-21 01:29:43 -07:00