First party sync service

[pruppert]

I’d like to stop using Dropbox since iCloud has replaced its function for me, but Alfred sync is on of the few things holding me back. I see it is not recommended to use iCloud Drive. Perhaps a new version of Alfred could have sync built in without any need for a third party like Dropbox or iCloud Drive? Thanks.

[dfay]

Why not just keep using Dropbox at the free level? 

[vitor]

10 hours ago, dfay said:

Why not just keep using Dropbox at the free level? 

 

It’s an extra app and service you have to worry about on your system. And it’s one that can at times be really resource intensive and has privacy implications (I remember something about them possibly scanning all your hard drive).

 

12 hours ago, pruppert said:

Perhaps a new version of Alfred could have sync built in

 

That seems unlikely, and I doubt you (or anyone else) would actually want that. It’d not only increase the cost of Alfred as an app, it would likely need to move to a subscription service. Alfred Preferences, since they include Workflows, can easily go for over 100MB (don’t even want to image for people who install Node Workflows). That amount of storage for each user, plus the overhead of coding and maintaining the sync server and maintaining said server would quickly shift Alfred’s focus. And how would you prevent abuse (people storing too much)? Now you either have people unhappy because they can’t sync everything, or you need to introduce paid tiers. The whole thing would quickly devolve into a mess not worth it for a two-person (one coder) team, seeing as the vast majority of users are happy with things the way they are.

 

12 hours ago, pruppert said:

I see it is not recommended to use iCloud Drive.

 

It’s not recommended because iCloud Drive has been unreliable for people in the past, going so far as to make them lose their preferences. But you aren’t forbidden from using it; you just need to be wary that if something goes wrong, you have been warned. If you’re using iCloud Drive as your primary source for everything else, you should be fine. Just remember to turn off the feature that deletes unused files from your machine (System Preferences → iCloud → iCloud Drive (Options…) → [unckeck] Optimize Mac Storage).

[pruppert]

@vitor, you have correctly summarized my reasons for wanting to move away from Dropbox. Your points on first party sync are also well made.

 

So is it your understanding that the iCloud issues were exclusively related to the Optimize Mac Storage settings? If that's the case, I'll probably go with iCloud since I have that setting disabled anyway.

[vitor]

5 minutes ago, pruppert said:

So is it your understanding that the iCloud issues were exclusively related to the Optimize Mac Storage settings?

 

Likely not exclusively. iCloud Drive was notoriously unreliable when it debuted. That said, I haven’t heard of any major issues for a good while. If that’s because it has improved or because most people stopped using it, I can’t say. Probably a bit of both.

 

In your shoes, I’d go for it but remember to backup everything once in a while.

 

That said, I do use iCloud Drive a lot with iA Writer, and haven’t had any trouble I recall (or that I noticed!).

[pruppert]

Yes, it seems to have improved a lot to me as well. I'll give it a go and have Arq running versioned backups on the Alfred folder in iCloud. Thanks for the help!

[pruppert]

Providing an update that I've been running my Alfred prefs in iCloud Drive for the past month without any issues.

[HillcroftDave]

Dropbox are now limiting "free" access to three devices - they haven't communicated this change to users at all!

 

If you are an existing user with more than three devices then you can continue, but you won't be able to add new devices including when you change devices (if that would take you over the limit).

 

https://help.dropbox.com/account/computer-limit

 

Dave

[sdd11]

On 2/1/2019 at 4:16 AM, pruppert said:

Providing an update that I've been running my Alfred prefs in iCloud Drive for the past month without any issues.

Do you turn off the Optimized Storage option?

[sdd11]

On 12/14/2018 at 10:46 PM, vitor said:

That seems unlikely, and I doubt you (or anyone else) would actually want that. It’d not only increase the cost of Alfred as an app, it would likely need to move to a subscription service. Alfred Preferences, since they include Workflows, can easily go for over 100MB (don’t even want to image for people who install Node Workflows). That amount of storage for each user, plus the overhead of coding and maintaining the sync server and maintaining said server would quickly shift Alfred’s focus. And how would you prevent abuse (people storing too much)? Now you either have people unhappy because they can’t sync everything, or you need to introduce paid tiers. The whole thing would quickly devolve into a mess not worth it for a two-person (one coder) team, seeing as the vast majority of users are happy with things the way they are.

Why this will increase the cost? Tons of app using Apple CloudKit to sync between devices. No need to maintain the sync server. The storage count towards user's iCloud drive storage, so there is no abuse issue either.

Edited March 17, 2019 by sdd11

[vitor]

1 hour ago, sdd11 said:

Tons of app using Apple CloudKit to sync between devices. No need to maintain the sync server. The storage count towards user's iCloud drive storage, so there is no abuse issue either.

 

Then it’s no longer a “first-party sync service”, which is the topic we’re discussing. There’s no point to building CloudKit integration; if you want to sync with iCloud (and have it count towards your iCloud data storage limit), then set your sync directory there. Alfred already supports what you’re suggesting, it just advises against it due to problems with the service in the past.

[sdd11]

13 hours ago, vitor said:

There’s no point to building CloudKit integration; if you want to sync with iCloud (and have it count towards your iCloud data storage limit), then set your sync directory there. Alfred already supports what you’re suggesting, it just advises against it due to problems with the service in the past.

Many apps using CloudKit work totally fine for me. I guess the reason why syncing file using iCloud is unreliable is that iCloud does not understand directory and file structure very well thus can not resolve conflict. By using CloudKit, developer can specify file infomation such as revision which can avoid conflict. Using CloudKit is much more user friendly. User does not need to set it up manually.

[vitor]

1 hour ago, sdd11 said:

Using CloudKit is much more user friendly. User does not need to set it up manually.

 

You’d still need to specify if you wanted to use CloudKit at all in this case, so you’d still need manual setup. Exact same number os steps.

Edited March 18, 2019 by vitor

[pruppert]

On 3/17/2019 at 6:35 AM, sdd11 said:

Do you turn off the Optimized Storage option?

 

Yes, I have that turned off.

[deanishe]

9 hours ago, sdd11 said:

developer can specify file infomation such as revision which can avoid conflict

 

CloudKit may work just fine for applications that manage their own data using its APIs, but that isn't Alfred.

 

A lot of the files being synced don't belong to Alfred: they belong to workflows. As a result, Alfred isn't in a position to manage (via revisions or whatever) the many sync conflicts iCloud's crapness causes because Alfred doesn't have sufficient information to do so.

 

Alfred's current sync Just Works with any software that can reliably sync files/directories, be it Dropbox, rsync, Unison, Seafile or whatever.

 

You're talking about completely rewriting Alfred's sync/file management in order to use a service that has a long history of losing data. Replacing a system that works wonderfully (if you're using reliable sync software).

 

The new limitations to Dropbox's free accounts suck, but I'm not convinced that's a sound reason to ditch a working, platform-agnostic sync solution for one that's tied to a legendarily unreliable service.

[sdd11]

14 hours ago, deanishe said:

 

CloudKit may work just fine for applications that manage their own data using its APIs, but that isn't Alfred.

 

A lot of the files being synced don't belong to Alfred: they belong to workflows. As a result, Alfred isn't in a position to manage (via revisions or whatever) the many sync conflicts iCloud's crapness causes because Alfred doesn't have sufficient information to do so.

 

Alfred's current sync Just Works with any software that can reliably sync files/directories, be it Dropbox, rsync, Unison, Seafile or whatever.

 

You're talking about completely rewriting Alfred's sync/file management in order to use a service that has a long history of losing data. Replacing a system that works wonderfully (if you're using reliable sync software).

 

The new limitations to Dropbox's free accounts suck, but I'm not convinced that's a sound reason to ditch a working, platform-agnostic sync solution for one that's tied to a legendarily unreliable service.

Make sense. Then I will use reliable sync software.

[deanishe]

FWIW, Seafile has worked well for me in the past. I don’t use it any more, though, because Dropbox handles symlinks better.

 

You need a server for Seafile, however. Syncthing is pure peer-to-peer and works very well. But without a server, the machines will only sync when both are online.

[Bruno Scheele]

 

On 3/19/2019 at 12:23 AM, deanishe said:

CloudKit may work just fine for applications that manage their own data using its APIs, but that isn't Alfred.

 

A lot of the files being synced don't belong to Alfred: they belong to workflows. As a result, Alfred isn't in a position to manage (via revisions or whatever) the many sync conflicts iCloud's crapness causes because Alfred doesn't have sufficient information to do so.

 

 

Deanishe, can you elaborate on this a bit more? I don't really understand what you mean with 'the files belong to workflows'.

 

* Is there something nice that Alfred is doing that allows Workflows to update files automatically from remote sources?

* Or are you talking about the user installing a new update to a workflow, which might update a large number of files and Alfred wouldn't know how to handle conflicts if the user happened to install another update on another machine, before it got synced automatically?

 

I'm in the same boat, where I want to drop Dropbox entirely, but Alfred is keeping me back. And from a iOS developer perspective, it's interesting to hear about why CloudKit is a bad solution for this.

[vitor]

38 minutes ago, Bruno Scheele said:

I want to drop Dropbox entirely, but Alfred is keeping me back.

 

You’re confusing CloudKit with iCloud Drive. CloudKit is something you use as the developer of an app when you know what will be stored. iCloud Drive is for the user to store anything. And since Alfred doesn’t know what will be in its preferences (because of Workflows), iCloud Drive is what makes sense.

If you want to use it, do.

[deanishe]

1 hour ago, Bruno Scheele said:

Deanishe, can you elaborate on this a bit more? I don't really understand what you mean with 'the files belong to workflows'.

 

Well, not just workflows. The way Alfred works is that it expects its preferences bundle to be changed by other applications. It could be Dropbox or Syncthing syncing your preferences, or npm symlinking a Node-based workflow, or a workflow saving/changing its own files. The filesystem is the source of truth, and Alfred monitors that.

 

CloudKit also expects to be the source of truth, and that your app's data will only be manipulated via the app's own CloudKit-aware APIs.

 

Getting Alfred and CloudKit to work together would require completely changing the way Alfred works or writing a custom filesystem–CloudKit sync engine.

Edited April 13, 2019 by deanishe

[Bruno Scheele]

1 hour ago, vitor said:

 

You’re confusing CloudKit with iCloud Drive.

 

No, I’m not? CloudKit is even mentioned in the quote I’m using.

 

But to alleviate confusion; as a customer the only thing I’m interested in is keeping a copy of my Alfred setup somewhere, so I’ll have access to it on new devices and/or on reinstalls. This is something that’s been done by other apps using either iCloud or CloudKit.

 

Currently Alfred only provides a file syncing version, for which it recommends Dropbox. iCloud is not recommended for good reasons.

 

However, Deanishe was mentioning specific reasons for why implementing CloudKit doesn’t make sense. However, I don’t understand the reasoning, so I asked clarification.

 

You’re giving the same reason, because of Workflows, Alfred doesn’t know what’s in its preferences. But I still don’t know why that’s the case; I have never touched the Alfred preferences folder directly, so Alfred should know exactly what’s in there. CloudKit would be a good fit for that case.

 

Hopefully that clears up the confusion. Hopefully someone can clarify why Workflows somehow wrests certainty from Alfred in such a way that CloudKit isn’t a good fit.

[deanishe]

17 minutes ago, Bruno Scheele said:

Hopefully someone can clarify why Workflows somehow wrests certainty from Alfred in such a way that CloudKit isn’t a good fit.

 

Did you read my last post?

[Bruno Scheele]

 

38 minutes ago, deanishe said:

 

Did you read my last post?

 

Just now. Apparently that got cross posted, so before I hadn’t.

 

Yup, your post clarifies it quite a bit. If the assumption is that anybody can edit that directory, then syncing is quite a bit harder.

 

Make me want to request CloudKit anyway, allowing me to just not care about other programs editing the preferences (I specifically don’t.)

 

Thanks for answering!

Edited April 13, 2019 by Bruno Scheele

[vitor]

49 minutes ago, Bruno Scheele said:

No, I’m not? CloudKit is even mentioned in the quote I’m using.

 

But not iCloud Drive, so that argument doesn’t hold. Furthermore, knowing two things exist doesn’t mean you understand the difference between them.

 

55 minutes ago, Bruno Scheele said:

iCloud is not recommended for good reasons.

 

iCloud Drive. “iCloud” is an umbrella term that includes iCloud Drive and CloudKit.

 

Yes, iCloud Drive isn’t recommended for a good reason. That good reason is that it was crap in the past, and it’s uncertain if all the kinks have been fixed. But it has gotten better. But CloudKit would be no more reliable than iCloud Drive, so arguing for one in detriment of the other doesn’t make sense.

 

Also proponents of CloudKit in this thread seem to be ignoring the fact that is has limits a developer has to pay to go over. With iCloud Drive, that responsibility is left to the user.

[deanishe]

1 hour ago, Bruno Scheele said:

Make me want to request CloudKit anyway, allowing me to just not care about other programs editing the preferences (I specifically don’t.)

 

You're talking about a massive re-write of the application to add a feature that would be broken for a significant number of users/workflows. Why would anyone even consider implementing that?