Apr 1, 2013
Alfred 2 came out a few weeks ago and I am loving the upgrade. The new workflow system is amazingly powerful and some of the things people are coming up with are great.
I have been playing around with building my own workflows since the release. Alfred's workflow system is very flexible in that it allows you to use whatever scripting language you are most comfortable with to generate the Feedback XML necessary for displaying your content in the Alfred pop-up. My language of choice is once again Python. Daniel Shannon has written alp, a python module intended speed up the development of Alfred workflows written in Python and it works very well.
Each week I check PreviewsWorld to see the comics that are coming out. I then add comics that I am interested in to my calendar. I've written an Alfred workflow to speed up this process.
The workflow takes two inputs: this and next. As you would expect, this returns this weeks comics while next returns next weeks comics.
Actioning on a publisher will take you to a list of the comics that publisher is releasing that week.
Actioning on a comic book will bring you to that comics description page on PreviewsWorld. Even more useful however is what happens when you hold command while actioning.
When command is held while actioning on a comic Fantastical is opened with the text field pre-populated with the comic and the date. If Fantastical is not installed, nothing will happen.
You can see the source and keep updated on the project over on Github. I encourage you to look into my use of autocomplete to assist in moving between pages in the workflow.
Writing this script has been a fun learning experience and I hope to make a few more of these workflows as I get the time. Let me know if you like it.
Jan 6, 2013
On my Macbook, I often run into links to specific posts on Twitter from sites such as Hacker News. However I'd prefer it if these links did not open in Safari, but Tweetbot instead. I wrote a quick script that does exactly that. Assign my script to a hotkey of your choosing using something like Keyboard Maestro and you are all set.
If you are viewing a person's Twitter profile and invoke the script it will open their profile in Tweetbot. Specific tweets will also open in Tweetbot.
Unfortunately, this python script relies on Applescript. If there is a more convienient way to obtain the URL of the currently active tab in Safari or to easily launch URLs I would greatly appreciate the information.
import os
from subprocess import Popen, PIPE
from urlparse import urlparse
# Set Default Browser
#browser = "WebKit"
browser = "Safari"
# Get url address from safari / webkit
cmd = "/usr/bin/osascript -e 'tell application \"%s\"' -e 'get URL of current tab of window 1' -e 'end tell'" % browser
pipe = Popen(cmd, shell=True, stdout=PIPE).stdout
url = pipe.readlines()[0].rstrip('\n')
# Get index of current tab
cmd = "/usr/bin/osascript -e 'tell application \"%s\"' -e 'get index of current tab of window 1' -e 'end tell'" % browser
pipe = Popen(cmd, shell=True, stdout=PIPE).stdout
tindex = pipe.readlines()[0].rstrip('\n')
# Parse URL
o = urlparse(url)
string = ''
if 'twitter' in o.netloc:
if o.path != '/':
if 'status' in o.path:
# Open specific tweet in tweetbot
string = "tweetbot:/%s" % o.path
else:
# Open user profile in tweetbot
string = "tweetbot:///user_profile/%s" % o.path
else:
print "Not a Twitter URL..."
# Open Twitter URL in Tweetbot
if string != '':
cmdList = []
# Send to tweetbot, close blank safari window, switch back to original tab
cmdList.append("""osascript -e 'tell application \"%s\" to open location \"%s\"'""" % (browser, string))
cmdList.append("""osascript -e 'tell application \"%s\" to close current tab of window 1'""" % browser)
cmdList.append("""osascript -e 'tell front window of application \"%s\" to set current tab to tab %s'""" % (browser, tindex))
# Execute Commands
for cmd in cmdList:
os.system(cmd)
Also on Github
Jul 7, 2012
Working on my Tweet Archiver python script has been very fun. I have learned a fair amount about the Tweepy module and how it interfaces with Twitter's API's. Dr. Drang's post on timezones was very informative, as I was experiencing UTC time zone issues myself. Pytz made dealing with these issues nearly painless. Rest assured that I'll find an infinite number of uses for these modules in the future.
The spirit of the inital conversation about archiving your tweets was clear:
- An archive of your tweets is something worth having.
- Twitter does not have an easy way to export your entire timeline.
- IFTTT provides a clear, and simple way to facilitate this archival process.
But as Dr. Drang put it perfectly in his article:
The questions I need to answer now are:
- Should I trust IFTTT to keep running?
- Should I continue to use ThinkUp as a backup?
- Should I just write my own script for archiving each day’s tweets so I don’t have to rely on ThinkUp or IFTTT?
Ignoring his comment about ThinkUp (I know next to nothing about this program), Dr. Drang brings up two interesting points. IFTTT is a great service, but for how long? If it ever does go down, will tweets that should have been archived be missed? Can I do this better myself?
I have come up with a script that replicates the original IFTTT recipe called 'autoTweetArchiver.py'. The code has been pasted below and is available on Github.
# autotweetArchiver.py
#
# Quickly archive your tweets to a plain text file.
# Attach this script to a cron task and automatically
# archive your tweets at any interval you choose!
#
# Created by: Tim Bueno
# Website: http://www.timbueno.com
#
# USAGE: python autoTweetArchiver.py {USERNAME} {LOG_FILE} {TIMEZONE}
#
# EXAMPLE: python autoTweetArchiver.py timbueno /Users/timbueno/Desktop/logDir/timbueno_twitter.txt US/Eastern
# EXAMPLE: python autoTweetArchiver.py BuenoDev /Users/timbueno/Desktop/logDir/buenodev_twitter.txt US/Eastern
#
# TODO:
import tweepy
import codecs
import os
import sys
import pytz
# USER INFO GATHERED FROM COMMAND LINE
theUserName = sys.argv[1]
archiveFile = sys.argv[2]
homeTZ = sys.argv[3]
homeTZ = pytz.timezone(homeTZ)
# lastTweetId file location
idFile = theUserName + '.tweetid'
pwd = os.path.dirname(__file__) # get script directory
idFile = os.path.join(pwd, idFile) # join dir and filename
# Instantiate time zone object
utc = pytz.utc
# Create Twitter API Object
api = tweepy.API()
# helpful variables
status_list = [] # Create empty list to hold statuses
cur_status_count = 0 # set current status count to zero
print "- - - - - - - - - - - - - - - - -"
print "autoTweetArchiver.py"
if os.path.exists(idFile):
# Get most recent tweet id from file
f = open(idFile, 'r')
idValue = f.read()
f.close()
idValue = int(idValue)
print "- - - - - - - - - - - - - - - - -"
print "tweetID file found! "
print "Latest Tweet ID: " +str(idValue)
print "Gathering unarchived tweets... "
# Get first page of unarchived statuses
statuses = api.user_timeline(count=200, include_rts=True, since_id=idValue, screen_name=theUserName)
# Get User information for display
if statuses != []:
theUser = statuses[0].author
total_status_count = theUser.statuses_count
while statuses != []:
cur_status_count = cur_status_count + len(statuses)
for status in statuses:
status_list.append(status)
theMaxId = statuses[-1].id
theMaxId = theMaxId - 1
# Get next page of unarchived statuses
statuses = api.user_timeline(count=200, include_rts=True, since_id=idValue, max_id=theMaxId, screen_name=theUserName)
else:
# Request first status page from twitter
statuses = api.user_timeline(count=200, include_rts=True, screen_name=theUserName)
# Get User information for display
theUser = statuses[0].author
total_status_count = theUser.statuses_count
print "- - - - - - - - - - - - - - - - -"
print "No tweetID file found..."
print "Creating a new archive file"
print "- - - - - - - - - - - - - - - - -"
while statuses != []:
cur_status_count = cur_status_count + len(statuses)
for status in statuses:
status_list.append(status)
# Get tweet id from last status in each page
theMaxId = statuses[-1].id
theMaxId = theMaxId - 1
# Get new page of statuses based on current id location
statuses = api.user_timeline(count=200, include_rts=True, max_id=theMaxId, screen_name=theUserName)
print "%d of %d tweets processed..." % (cur_status_count, total_status_count)
print "- - - - - - - - - - - - - - - - -"
# print "Total Statuses Retrieved: " + str(len(status_list))
print "Writing statuses to log file:"
#Write tweets to archive
if status_list != []:
print "Writing tweets to archive..."
print "Archive file:"
print archiveFile
print "- - - - - - - - - - - - - - - - -"
f = codecs.open(archiveFile, 'a', 'utf-8')
for status in reversed(status_list):
theTime = utc.localize(status.created_at).astimezone(homeTZ)
# Format your tweet archive here!
f.write(status.text + '\n')
f.write(theTime.strftime("%B %d, %Y at %I:%M%p\n"))
f.write('http://twitter.com/'+status.author.screen_name+'/status/'+str(status.id)+'\n')
f.write('- - - - - \n\n')
f.close()
# Write most recent tweet id to file for reuse
print "Saving last tweet id for later..."
f = open(idFile, 'w')
f.write(str(status_list[0].id))
f.close()
if status_list == []:
print "- - - - - - - - - - - - - - - - -"
print "No new tweets to archive!"
print "Total Statuses Retrieved: " + str(len(status_list))
print "Finished!"
print "- - - - - - - - - - - - - - - - -"
To run the script, clone the Github repository or copy and paste the code into your favorite text editor and save it to something like 'autoTweetArchiver.py'. Make sure you have the required dependencies installed, then execute it with:
python autoTweetArchiver.py {USERNAME} {LOG_FILE} {TIMEZONE}
Replace the bracketed arguments with your personal details. Make sure it looks something like this:
python autoTweetArchiver.py timbueno /logDir/twitter.txt US/Eastern
On first execution the script will
- Grab all of your previous tweets (up to 3,200)
- Write these tweets to the specified text file
- Create a 'username.tweetid' file in the same directory as the script
Upon successive executions the script will
- Open the 'username.tweetid' file and extract the most recent tweet archived
- Grab all tweets that have happened since last execution
- Append these new tweets to the end of the specified text file
- Update the 'username.tweetid' file for next execution
The 'username.tweetid' file simply contains the ID number twitter gives every tweet. I use this value to gather all tweets between last execution and the current one.
The third input argument, '{TIMEZONE}' must be formatted corectly or the script will fail. Tweepy's status object contains a python datetime object set to UTC time. The user must provide their timezone to localize the archive to their local time. For a list of available timezones head over here. If you're timezone is not listed there, let me know and I can add it.
Of course, no one wants to have to manually run this script. The beauty of the IFTTT recipe is that they handle the automation for you.
There are many different ways that the executions of this script can be automated, although none are as simple as using IFTTT. Macdrifter uses Keyboard Maestro for all of his automation tasks. This method is great for those of you wishing to run this script automatically on your local machine. Scripts can be set to run at any interval, with a very simple setup.
Apple provides the program launchd, however I have not had any experience with it, so I will not be explaining how to set this up.
Finally, my preferred method is to have cron handle the scheduling and automation. I have edited 'crontab' (with 'crontab -e') on my remote server with a line like this:
* * * * * python /home/blog/scripts/SimpleTweetArchiver/autoTweetArchiver.py timbueno /path/to/archive/Dropbox/logs/timbueno_twitter.txt US/Eastern
I have placed the 'timbueno_twitter.txt' archive text file in the Dropbox folder on my server. Saving the archive file to Dropbox allows me to access the archive from anywhere in the world.
The first five characters in the cronjob (' * * * ') tell cron to execute the following script every minute. This isn't very practical as it is unnecessary to update the archive that often. Below I have included common cron schedules:
Run once a year, midnight, Jan. 1st
0 0 1 1 *
Run once a month, midnight, first of month
0 0 1 * *
Run once a week, midnight on Sunday
0 0 * * 0
Run once a day, midnight
0 0 * * *
Run once an hour, beginning of hour
0 * * * *
Cron's syntax can be confusing so if you get stuck, head over here for some help.
You can archive multiple twitter accounts automatically as well. Add additional jobs to your 'crontab' to make it happen. Make sure each log file is named differently, or you will append the tweets of multiple twitter usernames to the same logfile (maybe you want that, go for it!).
The past few days have been an enjoyable learning experience for me, and I hope someone somewhere can take advantage of these scripts. I'll continue to work on them as time allows. As the source is freely available on Github I welcome everyone to pick apart my code, add features, log styles, and fix any pesky bugs you may find.
I've got more ideas for python scripting coming down the pipeline as well so check back here soon! test
Jul 5, 2012
Earlier this week I became interested in a discussion floating around the internet dealing with a very cool IFTTT recipe. IFTTT allows for easy scripting between some of your favorite web services. While the functionality is certaintly limited, creative individuals are cooking up cool uses for the service. This particular recipe archives every status update you send to twitter to an extremely light-weight plain text file. The plain text format is great as it will always be readable and is not bound to any particular service or app.
The only issue with the IFTTT recipe is that it will not archive any tweet that you posted before setting the recipe up, limiting its usefulness. Both Brett Terpstra and Dr. Drang created excellent solutions, but they assume that you have access to your old tweets in other locations, such as ThinkUp. While ThinkUp is nice, its not very practical to set up if you are only interested in using it once.
Today, I wrote a quick python script that grabs every tweet you have ever posted, and writes it to a file in the same format as the IFTTT recipe mentioned earlier. You can modify the output to fit any custom style fairly easily as well.
My current 'twitter.txt' archive file has each tweet formatted like this:
RT @kanyewest: I love me
July 31, 2010 at 05:57PM
http://twitter.com/timbueno/20008539872
You can find the Github page here. I have also posted the code (as it stands right now) below:
# tweetArchiver.py
#
# Quickly archive your tweets to a plain text file.
#
# This script is limited to retrieving only your
# 3,200 most recent tweets (you have twitter
# to thank for that)
#
# Created by: Tim Bueno
# Website: http://www.timbueno.com
#
# USAGE: python tweetArchiver.py
import tweepy
import codecs
import pytz
utc = pytz.utc
# Archive file location
archiveFile = "/Users/timbueno/Desktop/logDir/twitter.txt"
theUserName = 'timbueno'
homeTZ = pytz.timezone('US/Eastern')
# Create Twitter API Object
api = tweepy.API()
# helpful variables
status_list = [] # Create empty list to hold statuses
cur_status_count = 0
# Request first status page from twitter
statuses = api.user_timeline(count=200, include_rts=True, screen_name=theUserName)
# Get User information for display
theUser = statuses[0].author
total_status_count = theUser.statuses_count
print "- - - - - - - - - - - - - - - - -"
print "- Archiving "+theUser.name+"'s tweets"
print "- Archive file:"
print "- " + archiveFile
print "-"
print "- http://www.timbueno.com"
print "- - - - - - - - - - - - - - - - -"
while statuses != []:
cur_status_count = cur_status_count + len(statuses)
for status in statuses:
status_list.append(status)
# Get tweet id from last status in each page
theMaxId = statuses[-1].id
theMaxId = theMaxId - 1
# Get new page of statuses based on current id location
statuses = api.user_timeline(count=200, include_rts=True, max_id=theMaxId, screen_name=theUserName)
print "%d of %d tweets processed..." % (cur_status_count, total_status_count)
print "- - - - - - - - - - - - - - - - -"
print "Total Statuses Retrieved: " + str(len(status_list))
print "Writing statuses to log file:"
f = codecs.open(archiveFile, 'a', 'utf-8')
for status in reversed(status_list):
theTime = utc.localize(status.created_at).astimezone(homeTZ)
f.write(status.text + '\n')
f.write(theTime.strftime("%B %d, %Y at %I:%M%p\n"))
f.write('http://twitter.com/'+status.author.screen_name+'/status/'+str(status.id)+'\n')
f.write('- - - - - \n\n')
f.close()
print "Finished!"
print "- - - - - - - - - - - - - - - - -"
The "status_list" contains any number of tweepy Status object. This list is looped over and each status object is available to process in any way you choose. The code to where this data is formatted begins on line 57. The status object has a ton of content about each individual tweet. I've included an example structure of the status object below
{
'contributors': None,
'truncated': False,
'text': '@ERCourtney Working with anyone\u2019s code but your own is a very frustrating experience most of the time. I hear ya.',
'in_reply_to_status_id': None,
'id': 220264265720930304,
'_api': <tweepy.api.API object at 0x1096afc10>,
'author': <tweepy.models.User object at 0x10990ae10>,
'retweeted': False,
'coordinates': None,
'source': 'My Top Followers in 2010',
'in_reply_to_screen_name': None,
'id_str': '21041793667694593',
'retweet_count': 0,
'in_reply_to_user_id': None,
'favorited': False,
'source_url': 'http://mytopfollowersin2010.com',
'user': <tweepy.models.User object at 0x10990ae10>,
'geo': None,
'in_reply_to_user_id_str': None,
'created_at': datetime.datetime(2012, 7, 3, 21, 14, 27),
'in_reply_to_status_id_str': None,
'place': None
}
...and the tweepy User object for completness.
{
'follow_request_sent': False,
'profile_use_background_image': True,
'id': 132728535,
'_api': <tweepy.api.api object="" at="" xxxxxxx="">,
'verified': False,
'profile_sidebar_fill_color': 'C0DFEC',
'profile_text_color': '333333',
'followers_count': 80,
'protected': False,
'location': 'Seoul Korea',
'profile_background_color': '022330',
'id_str': '132728535',
'utc_offset': 32400,
'statuses_count': 742,
'description': "Cars, Musics, Games, Electronics, toys, food, etc... I'm just a typical boy!",
'friends_count': 133,
'profile_link_color': '0084B4',
'profile_image_url': 'http://a1.twimg.com/profile_images/1213351752/_2_2__normal.jpg',
'notifications': False,
'show_all_inline_media': False,
'geo_enabled': True,
'profile_background_image_url': 'http://a2.twimg.com/a/1294785484/images/themes/theme15/bg.png',
'screen_name': 'jaeeeee',
'lang': 'en',
'following': True,
'profile_background_tile': False,
'favourites_count': 2,
'name': 'Jae Jung Chung',
'url': 'http://www.carbonize.co.kr',
'created_at': datetime.datetime(2010, 4, 14, 1, 20, 45),
'contributors_enabled': False,
'time_zone': 'Seoul',
'profile_sidebar_border_color': 'a8c7f7',
'is_translator': False,
'listed_count': 2
}
Either clone my Github repository locally, or copy and paste the scripts code from above and save it to a file called something like 'tweetArchiver.py'. Make sure you have tweepy and pytz (for timezones correction) installed as well. It's easy if you have pip installed:
pip install tweepy
pip install pytz
Edit the 'archiveFile' variable at the top of the script to the file location you want to be written to:
archiveFile = "/Users/timbueno/Desktop/logDir/twitter.txt"
Execute my script like so:
:::bash
python tweetArchiver.py
After the log file is created you can point the aforementioned IFTTT recipe to the log file for further appending each time you tweet!
I cooked this up earlier today so there very well could be some bugs. Feel free to let me know of any you find!
Edit: As Brett Terpsta has pointed out, my script is limited to around 3200 statuses. The heavier twitter users, and the not so distant future me, will be over this limit. So if you are under 3200 tweets, now is the time to begin archiving!
Edit 2: Dr. Drang weighed in on my code as well. My original implementation required the user to authenticate their twitter acount with my application. This ultimately was not necessary and I have removed it from the code above and have pushed the changes to Github.
Edit 3: I hope this is the last edit! I've updated the tweetArchiver.py script once again to adjust for timezones. However, this did add a dependencey on the 'pytz' module. But if its good enough for Dr. Drang, its good enough for me!
Jun 27, 2012
A couple months back I quickly threw together an Applescript that allowed easy submission of links to Pinboard (my favorite bookmarking utility) from the convenience of Alfred (the do-pretty-much-everything app launcher). If you use both and ALSO happen to use Google Chrome on a computer that runs Apple's OS X, you will love the convenience. A long shot, I know.
To activate:
- Summon Alfred via your hot key
- Type “pin tag1 tag2 tag3 ...”
- Press Enter
- Growl will notify you if the submission was a success!
You must use tags for the script to work. Right now, I find it convenient that it forces me to tag my bookmarks correctly. However, I may look into removing this limitation in the future.
Download
I have included the Applescript below for anyone who wants to probe. I realize there is probably a quicker/better way to accomplish this but it works and I wrote it when I was just starting out with Applescript.
Currently the script only works in Google Chrome. I would love to bring functionality over to Safari and that may be something I look into now that Safari in Mountain Lion has an omnibar like Chrome's.
I have a few of these "convenience" scripts lying around that I will try and put on Github in the coming weeks.
--****************************************
-- Name: Chrome to Pinboard
-- Language: Applescript
-- Author: Tim Bueno
-- Website: http://www.timbueno.com
-- Date: 4/11/2011
--****************************************
on alfred_script(q)
try
set userName to "USERNAME_GOES_HERE"
set userPass to "PASSWORD_GOES_HERE"
set inputt to q as text
set tmp to splitString(inputt as text, " ")
set AppleScript's text item delimiters to "+"
set q to tmp as string
set AppleScript's text item delimiters to ""
tell application "Google Chrome"
set theURL to URL of active tab of first window
set theDesc to title of active tab of first window
end tell
set tmp to splitString(theDesc as text, " ")
set AppleScript's text item delimiters to "+"
set theDesc to tmp as string
set AppleScript's text item delimiters to space
set growlDesc to tmp as string
set AppleScript's text item delimiters to ""
set shellScript to ("curl --url \"https://" & userName & ":" & userPass & ¬
"@api.pinboard.in/v1/posts/add?url=" & theURL & "&description=" & theDesc & "&tags=" & q & "\"")
set PnbdResponse to (do shell script shellScript)
if PnbdResponse contains "code=\"done\"" then
tell application "Growl" to notify with name ¬
"Extension Output" title ¬
"Sent to Pinboard" description ¬
"\"" & growlDesc & "\" has been saved!" application name ¬
"Alfred" icon of application "Alfred"
else
tell application "Growl" to notify with name ¬
"Extension Output" title ¬
"Failed! - Sent to Pinboard" description ¬
"There was a problem. Check your username and password." application name ¬
"Alfred" icon of application "Alfred"
end if
end try
end alfred_script
--****************************************
-- Split String
--(thanks to Geert JM Vanderkelen for this code!
--http://geert.vanderkelen.org/post/241/)
--****************************************
to splitString(aString, delimiter)
set retVal to {}
set prevDelimiter to AppleScript's text item delimiters
log delimiter
set AppleScript's text item delimiters to {delimiter}
set retVal to every text item of aString
set AppleScript's text item delimiters to prevDelimiter
return retVal
end splitString