Stuck Sync/Too Many Requests

From Neos Wiki
Revision as of 04:55, 9 April 2023 by FuzzyBot (talk | contribs) (Updating to match new version of source page)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Other languages:

DO NOT

  • Delete your Neos data/database or cache files. Doing this will likely make you lose your unsynced items, and can cause other side effects
  • Try to save and sync any more items/worlds. Doing this will add to your sync queue and if your items/worlds are not syncing properly.
  • Click "Exit & Save Homes". Doing this will also add to your sync queue.
  • Send any objects or voice messages through the Contacts tab. Doing this will also add to your sync queue and will not send until any syncing issues are fixed.
  • Change any settings. Doing this will also add to your sync queue and will not save properly until any syncing issues are fixed.
  • Reinstall Neos. Doing this does not help syncing issues, as these are stored in a separate location.

Error Description

If you're here, then you've found a Stuck Sync that is caused by the "TooManyRequests" error.

You've found a message that contains something like this:

Fail Reason: Response status code does not indicate success: TooManyRequests (429); Substatus: 3200; ActivityId: ....; Reason: (
code : TooManyRequests
message : The request rate is too large. Please retry after sometime. Learn more: http://aka.ms/cosmosdb-error-429, 

This error means that there's a rate limit occurring between you and our data storage systems which are a part of the Azure cloud service offerings. This usually happens when items are being saved too quickly or are too large to save temporarily. The issue usually goes away on it's own if you are patient.

However, You can follow these steps to try and resolve it:

  1. Try restarting Neos, this will re-start the sync and try it again.
  2. Try waiting for a while before restarting or trying to sync again.
  3. Try waiting for a quieter time for Neos traffic before syncing. Later at night or earlier in the morning

This may take up to 48 hours to resolve.

If you are impatient, then there are some more items you can try. These WILL CAUSE DATA LOSS:

  • Force the Sync to ignore records which are causing it to get stuck. This is done by following the Sync Conflict guide and using the -DeleteUnsyncedCloudRecords option within this guide.
  • You can Clear your database.

Avoiding this Error

The Neos team are aware and are investigating this issue, see the bug report on our issue tracker for more information. In the mean time, try these recommendations to potentially reduce the amount of times you see it:

  1. Avoid repeatedly saving the same object multiple times - For example, waiting a longer period of time between saving copies of worlds or avatars.
  2. Avoid saving multiple items at once - For example, don't grab and save every single object you see.
  3. Avoid saving very large objects, for example a very complicated world, instead save portions of the object such as individual rooms or objects separately.
  4. Break up your saves, save an item...wait 5 minutes... save another item.
  5. Try using your Local Home as a Local Backup.
  6. Continue posting reports, log files and information on the bug report on our issue tracker
  7. Ask in our discord for more information.

Large Items

We've also seen reports of this occurring with exceptionally large assets. For example very high poly scenes and worlds, anatomically correct research models etc. In these cases these assets contain so much information that they can exceed the rate limiting just by themselves. If this is the case, then please try breaking them up as recommended above.