Copycopter slows down development and staging environment to the point that it's unusable

[aaronjensen]

It takes anywhere from 3-7 seconds usually for copycopter to check for updates to strings. I don't know if it's because our app has grown to the point where the dump is several MB or what, but it makes the development environment unusable. Furthermore, in order to disable the refresh we have to set the development environments to [] which has the unfortunate side effect of not updating the server whenever we add a new key. So we really need to manually switch between running in "development" (according to copycopter) and not.

Is there a way to speed things up and/or have a hybrid environment in which new strings will get pushed to the server but it won't check for updates every request?

[jasonm]

Hi Aaron,

Is this in a single request to your server, or to a set of requests (including assets). The reason I ask is that, in Rails 3.1, the asset requests are handled by Sprockets instead of just ActionDispatch::Static, and the copycopter_client fetches new blurbs in each non-static request, making the asset requests very slow. That's an issue we definitely want to address.

If that's not the case, is it consistently 3-7 seconds, e.g. an intermittent network issue? I'm curious if your application's copy content is much larger than typical. Can you let us know your account so we can take a look? Feel free to open a private issue on http://help.copycopter.com/ if you'd rather continue the discussion there.

In the meantime, the RequestSync middleware (which syncs copy on each request) is only enabled for environments included in CopycopterClient::Configuration#development_environments, which you could override in your config/initializers/copycopter.rb as:

CopycopterClient.configure do |config|
  config.api_key = 'key-goes-here'
  config.development_environments = %w(staging) # defaults to %w(development staging)
end

Thanks,
-Jason

[aaronjensen]

Hi Jason,

We're on a Rails 3.0.X app. It's the main request that's taking 3-7 seconds every time.

We have tried setting development_environments. As I mentioned in my initial ticket, it seems to have the side effect of not updating the server when new keys are added. Furthermore, unless I misunderstood the code, this also causes it to show published strings instead of draft strings. This doesn't work very well on the staging server.

In other words, we need to toggle between including development and not depending on what we're working on.

It would be nice to have more granular settings for these things in each environment. Ideally, until the sync time is sped up greatly, we would want to be able to set it up so Staging used 5min sync instead of request but still used draft, and development used 5 minute sync (or on demand?), draft, and uploaded new keys/defaults.

Thanks,

Aaron

[jasonm]

Hi Aaron,

Thanks. I'd recommend two changes - one that can be done with the configuration, the other which needs code changes to the copycopter_client. For the code changes, I'd like @jferris opinion on the API before we make the change in the released gem, but you can fork the gem and patch it in the meantime to keep your development and staging speedy.

My general approach would be to leave the development_environments setting as default, disable the RequestSync, and rely solely on the Poller to fetch new content, but with a much lower polling interval.

You can set the polling delay through configuration:

CopycopterClient.configure do |config|
  config.api_key = 'key-goes-here'
  config.polling_delay = 5 # seconds, defaults to 300 seconds or 5 minutes
end

Disabling RequestSync is currently tied to whether your Rails env is considered a development environment. Since there are other concerns tied to running in a development environment (specifically draft content vs published content), I think we should add another configuration option to allow explicitly disabling the RequestSync. It seems to me that if you want to disable RequestSync in any development environment, you would want to disable it for all development environments (e.g. if dev is slow, staging is also slow).

There's no config option for this presently, so I'd recommend forking the gem and removing/commenting:

https://github.com/thoughtbot/copycopter_client/blob/b74ad613c6/lib/copycopter_client/configuration.rb#L175

Eventually, there could be a config.disable_request_sync = true option. I'm open to other API suggestions, too.

Thanks,
-Jason

[aaronjensen]

Thanks Jason. Rather than forking and making that change I'm trying this out:

CopycopterClient.configure do |config|
  config.api_key = ''
  config.http_read_timeout = 60
  config.http_open_timeout = 60
  config.polling_delay = 10 unless Rails.env.production? # seconds, defaults to 300 seconds or 5 minutes
  config.middleware = nil
end

Setting middleware to nil seems to have the same effect. I also included the unless Rails.env.production? so that the production server wouldn't pound copycopter.

[croaky]

@aaronjensen This is now fixed in 1.1.2. You should be able to update the gem. I've been keeping my timeouts at 30 seconds just in case, though.

https://github.com/thoughtbot/copycopter_client/commit/fa1ed25085f164c4cc25495e31a16eb227e77061

[aaronjensen]

This fix is great, but wasn't my problem. My problem is that devise uploaded 9 or so additional languages and my strings file is up to 2.5MB. downloading a 2.5MB from copycopter every request is the problem.

[jferris]

@aaronjensen We're working on ways to deal with the larger download sizes and unused languages. In the meantime, I recommend you keep your configuration workaround in place. Thanks for your patience.

[semanticart]

Does anyone (new maintainers or old) have any new thoughts on this? I want to use copycopter, but devise and simple_form-related translation bloat does indeed make this unusable. I'm sitting on 5.08mb of json for the draft_blurbs.

The etags obviously help, but things still choke up often.

I'd love a way to specify that i only want certain locales (the app I'm currently working on is US-centric). In the meantime, I've added some code to my copycopter server to add where("locales.key = 'en'") to the blurb.to_hash method. Despite being an unfortunate hack, this cuts the json down to 223kb.

Any thoughts are appreciated. I'm happy to pitch in and work on a pull request if someone wants to point me in the right direction. Thanks.

[jferris]

My plan was to add a locale manager to the UI that would let users select which locales should be downloaded. I also considered an option in the client to prevent some locales from being uploaded in the first place.

Another option would be to have locales "activate" in the client once they were actually requested by the application. That way you could automatically decide which locales to send to the server based on which ones were in use.

[kwerle]

A locale manager seems like a good fix. Automagically adding a locale is flashy and all, but not even remotely useful given that it seems to also lead to locale-bloat.

[p1nox]

Have you tried to use copycopter only for development environment ( https://gist.github.com/p1nox/4755262 ) ? It could be an easy workaround... What do you think?