Jump to content
rice.shawn

Packal: Workflow and Theme Repository

Recommended Posts

The update feature has always been planned and was about 90% written before Packal.org went live. You can read this post about the update workflow. If you open the folder for any workflow downloaded from Packal, you'll see a "packal" folder inside it that has metadata in it as well as a code signature. These are there in order to facilitate the upgrading.

 

I had many, many conversations with Andrew developing the update mechanism because I really wanted to address all security concerns in the migration process. He liked a solution that I eventually came up with, and, so, the updating process will differ from Alleyoop in that it won't just download the workflow, but it will actually upgrade the workflow.

 

It won't be an automated thing because that would be very un-Alfred like, so you will have to invoke the workflow to check for updates.

 

 

I'm creating a pretty granular update system to make sure that the user has full control over what Packal does and doesn't update (for instance, @smarg19, it shouldn't update ZotQuery for you, but only for you), and so I've created a pretty complex GUI that mirrors the app store. This will be used for configuration purposes, but users can also update workflows from there in the same way the app store lets you do so.

 

After it is configured, however, you'll be able to do updates from Alfred without the GUI if you so prefer.

 

The code status is that most of it is written, but I need to polish the GUI, and I need to stitch together all the updating logic. Each piece has been written, but they need to be put together, which will, of course, make everything break, and so I'll have to fix that quite a bit.

 

Work on the workflow has been temporarily paused because of dissertation deadlines and a few other exigent projects in life, but it should resume soon.

 

Another adjunct workflow is being developed to allow workflow contributors to push updates to packal from Alfred itself. You'll still have to fill out the form to submit new ones, but you'll be able to update some of the documentation fields and the workflow itself from a few keystrokes in Alfred. Dean and Richard are helping with that. Right now, we have a very nascent API on a dev server that needs to be extended before there is any use.

 

So, these are just updates that I hope will become available soon, and I'll push work on them as well.

Share this post


Link to post

Oh, and you can see from the pictures on the post that it also keeps backups of the workflows when updating (in case something goes wrong), and there is opt-out usage data sent to the server. You can see the pictures of the preliminary GUI there.

Share this post


Link to post

Simply as a thought-experiment, would it be possible/helpful/permissible to add some sort of update checking into Packal workflows? 

 

Here's one possible implementation: Use a Outputs -> Run Script which all of your workflow actions pipe into. This script will make a call to the Packal page for that workflow (use some sort of id for this). Packal will return a simple JSON string, which will include the version number of the workflow housed on Packal. If that number differs from the version being used, then prompt a reminder to update. Clearly this would require a JSON file in the workflow which holds the installed workflow's version. 

 

Now I believe this is similar to the now-defunct Alleyoop set-up. I started writing workflows after this system died, so I've never used it, but it doesn't seem *too* difficult to implement something like the above to make it easier to know when to update. 

 

I press this issue as someone who has a complex workflow that had fairly limiting bugs in early versions. When I fix a bug, I want everyone to have the new, better version. And I can't trust everyone to regularly check for updates. It just seems like a highly beneficial feature. 

 

That would put massive load on the Packal server. You can't have workflows hitting the API every time they're run.

 

Also, you don't need Packal to do that. Just store a file called version (or version.json or whatever) in your workflow and retrieve the file from your Github repository (instead of hitting the Packal API). If the versions differ, notify the user of the update.

Share this post


Link to post

Oh, and you can see from the pictures on the post that it also keeps backups of the workflows when updating (in case something goes wrong), and there is opt-out usage data sent to the server. You can see the pictures of the preliminary GUI there.

 

What do you think about dumping a list of Packal workflows as a JSON file to the Github repo where the workflow files are stored? It could be pretty simple (workflow name, bundle ID, version, Packal URL) and updated every hour or two via a cron job.

 

It would probably save load on the server vs having the workflow hit the Packal server, and might prove useful for other users.

 

If it also included workflow author info, it could serve as the basis for the upload workflow, too.

Edited by deanishe

Share this post


Link to post

I'm so totally ahead of you.

 

All Packal workflows are hosted on Github and downloaded from there. The download tracker is just embedded in the link. Whenever someone uploads a new workflow file (update or new workflow entirely), then the manifest (XML) that lives in the repo is recreated. Look at the URLs on the download links from Packal. The entire update mechanism works with Github and not Packal itself, and this is fine because everything in the repo has been processed and pushed via Packal, ensuring its integrity.

 

Where Packal does (or will) excel is that it pulls in a lot of security aspects for updating and keeping track of updates without the workflow author having to do anything to make it happen. All workflows are tested for integrity, subjected to a virus scan, and code signed. In the updating process, the code signing is checked so that the packages are authenticated as coming from the original workflow author via Packal. It's safe, secure, and simple.

 

I don't want to use anything like cron.d or an upstart / init job. While those would be easy to create, they would require an uninstaller that couldn't be invoked automatically (well, there are hacky ways to do ensure that) if the workflow is deleted from the users' system. I fully believe that everything should be kept in Alfred itself, and, the way the workflow is setup, to check for updates you just bring up Alfred and start to type "packal," and the manifest is updated if it falls outside the cache window (just a couple of hours). After that, you can just click "enter" (if you'd like) to update all of them. I know that it requires a step for the user, but it isn't a hard step to do, and automating things through any of the above mentioned ones bring in more pros than cons.

 

Since the entire repo is on Github, anyone can clone it, but, as I mention on the FAQ page, it seems to defeat the purpose of Packal in that the site helps you find relevant workflows. There are so many great ones, but I don't have a use for many of them (but others do).

 

The need for the GUI, at least for setting things up on that granular way, is that many people, especially me, download way too many workflows, and trying to put them all in Alfred just doesn't display the information effectively. I'm pretty sure that some people, as people do, will want to stick with an older version of a workflow for some random reason, and so those should be "blacklisted" (not checked for updates).

 

I might have some time to work on the update workflow this weekend. Maybe, just maybe, I could push a beta then.

Share this post


Link to post

All this sounds great. I look forward to the beta in the future (no rush or pressure from me). This will be a great boon for the Alfred community.

Share this post


Link to post

Just a fun update: Packal currently is hosting 98 workflows and 19 themes. Just two workflows away from the 100 mark! Anyone want to fix that?

 

I also want to thank everyone who has contributed to Packal so far. You're the ones who make the site valuable.

Share this post


Link to post

Hi Shawn

 

Any plans for commenting?

 

I'd love to be able to thank some of the workflow authors (eg this guy who made the OTP token generator http://www.packal.org/workflow/gauth-google-authenticator)

 

I made a small donation to the author but commenting on the workflow would be nice.

 

Voting or like buttons would be great too, and options to sort on those, along with downloads etc - though I'm sure this is all in your todo already

Edited by subvertbeats

Share this post


Link to post

No plans for comments. They cause more problems than they're worth often. If you want to reach any of the workflow authors, just click on their name/picture on the workflow page, and you'll be taken to their profile. There you might find their websites, alfred forum profile, github profile, or donation widget (that goes to the workflow author for proper thank yous, not packal). If you click through their websites or other profiles, you can probably find a way to send them a personal note and say, "this rocks," which would probably mean a lot more than a comment. Personal notes can be great uplifters.

Share this post


Link to post

Thanks Shawn

 

As mentioned I sent a donation, and added a personal comment to that.

 

By the way, not sure if its anything on your side or not, but I had to make the donation manually through the PayPal site - whenever I tried to donate using the link on the authors page, I'd get an error about needing to add funds to my account before I can send any more money - despite having funds available.

Share this post


Link to post

By the way, not sure if its anything on your side or not, but I had to make the donation manually through the PayPal site - whenever I tried to donate using the link on the authors page, I'd get an error about needing to add funds to my account before I can send any more money - despite having funds available.

 

That is really strange. The way the site is setup is just that it takes the email address that the user signed up with and turns it into a paypal widget by just taking the stock paypal widget code and dropping in their email address, so I'm not sure how that would throw the error that you're got. I'll have to think about it more, but that is really weird.

 

And thank you for sending the author a donation. I'm sure that it was much appreciated.

 

I forgot to mention that I am collecting download statistics, and, when I get the time, I'll integrate those into the site to allow for a "most popular" sort option. I had created a "liking" system a while ago, but it slowed down the loading time of the site much more than it was worth having, and the implementation of it didn't seem too feasible. I've found the site an interesting design challenge when it comes to trying to keep it lean yet still include necessary and good features. I still conceive of it as a searchable repository / display site for people's work but one that exists like that only in relation to these forums where the interaction / support aspect can take place. Without these forums, the site would look a lot different.

Share this post


Link to post

Can you sign up for the beta or is there a mailing list or other way to get notified when the beta is released?

 

Well, the "beta" tag is somewhat strange because the site is open and public, but it still needs some spit and polish (I'm usually a perfectionist, so letting people see it as is was hard for me to do). By that, I mean you'll probably run into some strange behavior from time to time, and the display isn't perfect, but it's good enough to be used and requires an occasional bit of patience. I went ahead and released it because I realized that the time window that I had for working on it was disappearing, and it was still a valuable resource as is. When I get the site more polished and some other sections crafted properly (especially the 'documentation' section, the dashboards for workflow/theme contributors, better searching/tagging, and the better 'popular' sorts), then I'll remove the "beta" tag from the site.

 

However, if you're referring to the "update" workflow, then there isn't a notification list or anything like that. When the first version is released, then you can expect to see a thread open up on the forums, a post on this thread, and a conspicuous, shiny, new notification on Packal itself. Since that workflow will manipulate data on your computer itself, it needs to be as bug-free as possible before anyone sees it.

Share this post


Link to post

For a bit of fun (and to test a Workflow library I'm working on), I made a Workflow to search Packal.

 

Tk7yOwy.png

You can get the Workflow here. The very brief source code is here.

It uses the manifest.xml file exported to Packal's GitHub repo, so the information available to the Workflow is very limited (e.g. no workflow URL and, alas, no tags). Hitting ENTER on a result will send you to Packal's search page, which should show you only one result: the Workflow you actioned.

As I say, it's just a bit of fun, so don't be asking for any funky features (though I'd happily add tag search and a proper URL if they're added to the manifest file, hint hint Shawn  ;)).

Share this post


Link to post

Hi,

 

Awesome idea, I just wish I could register... WTF is wrong with the Sign in form :

 

http://screencast.com/t/f2wmxTdkra

 

My email is perfectly valid but the form won't submit... Hopefully it'll be solved soon I have some cool workflow to share !!!

 

Cheers,

 

This is an error that is truly stumping me. I've been combing through logs and trying other diagnostics, but I'm not sure what would be making the error happen. But the error is stranger because it's a false positive not just in the sense that it's a valid email address, but that it actually creates the account.

 

I just checked, and there is an account for ben74 made about 2.5 hours ago.

 

I played with a few more settings, but I might have to dig into the validation settings themselves in order to figure out why the error popped up.

Share this post


Link to post

Hi,

 

Awesome idea, I just wish I could register... WTF is wrong with the Sign in form :

 

http://screencast.com/t/f2wmxTdkra

 

My email is perfectly valid but the form won't submit... Hopefully it'll be solved soon I have some cool workflow to share !!!

 

Cheers,

 

This is an error that is truly stumping me. I've been combing through logs and trying other diagnostics, but I'm not sure what would be making the error happen. But the error is stranger because it's a false positive not just in the sense that it's a valid email address, but that it actually creates the account.

 

I just checked, and there is an account for ben74 made about 2.5 hours ago.

 

I played with a few more settings, but I might have to dig into the validation settings themselves in order to figure out why the error popped up.

I'm getting this error too. I'm able to "reset" my password and log in via the one-time login link, but once I change my password I get back to this error. Although, I'm not convinced it's actually accepting my new changed password as there's no notification given that it has changed - it just loops back to the same form. Could you maybe try deleting my account altogether (CarlosNZ) and I'll start again with a fresh one? Thanks.

Share this post


Link to post

That's a really strange error. I'll try to figure out what's going on with it. Anyway, I did delete the account, so you can start again. Email me if you run into these sorts of issues again.

Share this post


Link to post

That's a really strange error. I'll try to figure out what's going on with it. Anyway, I did delete the account, so you can start again. Email me if you run into these sorts of issues again.

Hi Shawn, I think Firefox is causing the problem. When I use it, after clicking "Submit" on the registration page, I just get the same page reloading with no indication as to whether or not the account has been created. (It would now appear that it hasn't.) However, I just signed up again in Safari and it seems to be working fine so far.

 

Hope that helps.

 

C

Share this post


Link to post

Couple of quick questions about Packal:

  1. Is there any way to get download stats for one's own workflows? I'd like to gauge how much interest there is in my various workflows. (With my own downloads, I track numbers with bitly shortlinks.)
  2. Is there, or will there be, an RSS feed for the site to keep track of new submissions?

Thanks.

Carl.

Share this post


Link to post

Couple of quick questions about Packal:

  1. Is there any way to get download stats for one's own workflows? I'd like to gauge how much interest there is in my various workflows. (With my own downloads, I track numbers with bitly shortlinks.)
  2. Is there, or will there be, an RSS feed for the site to keep track of new submissions?

Thanks.

Carl.

 

Re: #1 — Not yet. Packal is collecting download statistics, but I haven't written a way for workflow authors to see them. That is on the todo list. Also, the Packal workflow has a "reporting" mechanism (which can easily be turned off) to send workflow information back to the Packal server. Basically, it sends a report approximately 1/week about which workflows are installed and which of those are enabled/disabled. I figure that information could be great for workflow developers to have. That functionality is mostly written, so it should be included in the initial release.

 

Re: #2 — Not yet, but good idea. I'll put that on the todo list and let everyone know when it's there.

 

One new thing about Packal, though: when you release a workflow, you can now specify a Github Repo if you use one, and the workflow page will link to the repo.

Share this post


Link to post

For anyone who, like me, was getting timeout errors on Packal, the site is rock-solid now.

Great work, Shawn.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×