Prowl notifications for Google Voice

An iPhone client for Growl available on the App Store.
kjconrad
Muffin
Posts: 30
Joined: Sat Jul 18, 2009 5:41 am

Re: Prowl notifications for Google Voice

Postby kjconrad » Thu Aug 13, 2009 12:36 am

Is the fix 2 posts up working for anyone? I keep getting the error I posted a couple lines up... What am I doing wrong?

edit: FOund out the error was because I had an txt message with a ™ in it. I deleted that from good voice and all is working fine now... THANKS!!!

hot_damn
Harmless
Posts: 6
Joined: Mon Aug 03, 2009 5:45 am

Re: Prowl notifications for Google Voice

Postby hot_damn » Thu Aug 13, 2009 4:42 am

We're up and running as well! Thanks so much.

The trouble-shooting took way longer than it should have, because we simultaneously found that T-mobile has blocked SMS to google voice accounts (at least in the 415 area code). I hate telecoms.

* sigh *

atomjack
Harmless
Posts: 6
Joined: Wed Aug 12, 2009 6:03 pm

Re: Prowl notifications for Google Voice

Postby atomjack » Thu Aug 13, 2009 5:06 am

I took another look at these scripts and did some modifications I thought I'd share with you guys. First, I figured out why it was re-pushing sms's when you reran the script - the cache file wasn't being loaded properly. I also went ahead and added support for voicemail notifications. It pretty much works the same as with sms's, and it does use the transcription of the voicemail, for the body of the notification.

You can download my modification here: http://bantha-fodder.com/files/googlevoicenotify.zip

It's just an update of googlevoicenotify.py. I'll email mike and maybe he'll want to put my changes in to his source on git.

Let me know what you guys think.

mikeyk
Harmless
Posts: 3
Joined: Thu Jul 16, 2009 11:38 am

Re: Prowl notifications for Google Voice

Postby mikeyk » Thu Aug 13, 2009 8:36 am

atomjack's changes are awesome and I've incorporated them into the latest release at:

http://github.com/mikeyk/googlevoicenotify/tree/master

Hooray for open source :)

bratwick
Harmless
Posts: 2
Joined: Wed Aug 12, 2009 10:18 am

Re: Prowl notifications for Google Voice

Postby bratwick » Thu Aug 13, 2009 3:31 pm

This is so cool! Thanks guys for all your hard work.

I've managed to get the various versions of the scripts to work on a couple of different linux servers (fedora 7 and ubuntu jaunty) and found that regardless of the configuration selections, the script stops running after a short, indeterminate period. I've tried using the bash script provided by Mike to run the script and I've also invoked it directly from python and backgrounded it - and still it quits after a short time (usually 5 to 10 minutes or less).

One thing I've had to do (including in the updated scripts) is change the "sleep_time" argument to the sleep function in prowlgooglevoice.py (line 66) to a number, since if I leave it as "sleep_time" the scripts will not run and I sometimes get an error saying something to the effect of "invalid argument - float expected." Changing the argument seems to allow the script to run - if only for a few minutes.

I might also note a couple other changes I had to make: 1) in the bash script (from Mike's tools folder on github), I changed the variable names assigned to the mustHaves variable on line 114 to match my file names (.gvnotify and .prowlapi instead of credentials and prowlkey) and I moved these two files into into the same directory as the scripts. Since one of the scripts looks for these files in one location and another expects them elsewhere, I thought this solution was as good as any.

I'm not sure these minor changes have anything to do with my scripts quitting - but I'm also not very familiar with python. Anybody having the same problem with scripts dying?

Either way, very exciting project. Keep up the good work!

hot_damn
Harmless
Posts: 6
Joined: Mon Aug 03, 2009 5:45 am

Re: Prowl notifications for Google Voice

Postby hot_damn » Thu Aug 13, 2009 5:18 pm

my live-in sys admin has told me he's using daemontools to keep the script running if it craps out. Here's what he sent me:

"http://cr.yp.to/daemontools.html

On many modern *n*xes you'll need this patch:

http://www.qmail.org/rpms/patches/daemo ... rrno.patch"

kjconrad
Muffin
Posts: 30
Joined: Sat Jul 18, 2009 5:41 am

Re: Prowl notifications for Google Voice

Postby kjconrad » Thu Aug 13, 2009 7:57 pm

I was also having problem with this script erroring out but have found a work around for myself. I'm running the python scripts and made the following shell script to help me out:

Code: Select all

#! /bin/sh
cd ~/googlenotifications/
python gmailnotification.py
./gv.command


The shell script is named "gv.command" which you will want to chmod +x. You'll also need to point it to the right directory obviously. It will start the gmailnotification.py and after it fails relaunch itself. A handy infinite loop incase the other infinite loop errors out :)

Also made one for the gmail script since I was also having problems with it erroring out.

bratwick
Harmless
Posts: 2
Joined: Wed Aug 12, 2009 10:18 am

Re: Prowl notifications for Google Voice

Postby bratwick » Fri Aug 14, 2009 7:29 am

Thanks folks. I'll try your suggestions. I had thought of running a script to respawn prowlgooglevoice (or the bash script that calls it) every so often, but it seemed to me I would have gotten duplicates of messages I hadn't marked "read" - over and over again - presumably every 5 or 10 minutes when the script was restarted. I think the new version addresses that. Cool beans.

@kj
Don't sweat Gmail notifications - as soon as GPush is available on the app store, you'll be able to offload that task for $.99.

kjconrad
Muffin
Posts: 30
Joined: Sat Jul 18, 2009 5:41 am

Re: Prowl notifications for Google Voice

Postby kjconrad » Fri Aug 14, 2009 3:46 pm

The new script is great, I've turned off forwarding of voicemails to e-mail on Google Voice and I'm not relying on this script. Question, For each voice mail I get 2 notifications. The first just tells me that I have a new voice mail and who it's from. Then a minute later I'll get another notification with the transcription. Is there a way to only get the notification with the transcription in it?


@bratwick
GPush sounds cool if all you need is GMail push notifications. But why give my username and password (and $0.99) to that service when I already have prowl and my own computer to push GV Notifications to me? I've been running the Gmail script for the last few weeks in a looped shell script with no problems, (as well as MikeK's script).

atomjack
Harmless
Posts: 6
Joined: Wed Aug 12, 2009 6:03 pm

Re: Prowl notifications for Google Voice

Postby atomjack » Sat Aug 15, 2009 3:22 am

kjconrad wrote:The new script is great, I've turned off forwarding of voicemails to e-mail on Google Voice and I'm not relying on this script. Question, For each voice mail I get 2 notifications. The first just tells me that I have a new voice mail and who it's from. Then a minute later I'll get another notification with the transcription. Is there a way to only get the notification with the transcription in it?

Hmm that's weird. Did you try it again? Make sure you clear the cache file so you get the initial notification of the voicemail, but in my testing I haven't had any voicemail notifications without the transcript. Is anyone else getting this behavior?

kjconrad
Muffin
Posts: 30
Joined: Sat Jul 18, 2009 5:41 am

Re: Prowl notifications for Google Voice

Postby kjconrad » Sat Aug 15, 2009 1:44 pm

atomjack wrote:
kjconrad wrote:The new script is great, I've turned off forwarding of voicemails to e-mail on Google Voice and I'm not relying on this script. Question, For each voice mail I get 2 notifications. The first just tells me that I have a new voice mail and who it's from. Then a minute later I'll get another notification with the transcription. Is there a way to only get the notification with the transcription in it?

Hmm that's weird. Did you try it again? Make sure you clear the cache file so you get the initial notification of the voicemail, but in my testing I haven't had any voicemail notifications without the transcript. Is anyone else getting this behavior?


Yeah it's happened with 2 voicemails so far (out of 2). I do have a 60 second refresh rate sent, I'm assuming it's picking it before the transcription hits, then a minute later again with the transcription.

It's funny because the transcription I get isn't as complete as the one I get if I wait a few minutes and then check the google voice page. Not to say the end is missing, I know we're limited to how many character can be in a push notification. It's more like words in the middle are missing, or wrong, and the google adds them in or corrects them. Google must add to the transcription as they analyze it.

atomjack
Harmless
Posts: 6
Joined: Wed Aug 12, 2009 6:03 pm

Re: Prowl notifications for Google Voice

Postby atomjack » Sun Aug 16, 2009 6:48 pm

Ok yeah, that makes sense, actually. When the script first picks up the voicemail, before the transcript, it caches the voicemail on your server (without the transcription), and then when the transcript comes through, it thinks the voicemail is a completely new one (it's using the from name/number + the transcription text as a unique identifier). I've emailed Mike to see what he thinks of a fix I came up with and I'll post again once he implements it, if he doesn't think there's any problems.

hot_damn
Harmless
Posts: 6
Joined: Mon Aug 03, 2009 5:45 am

Re: Prowl notifications for Google Voice

Postby hot_damn » Sun Aug 16, 2009 6:51 pm

I'm actually experiencing a flood of notifications when I receive two SMS's in quick succession. One prowl ping after another, actually interfering with use of my phone, until I mark them as read. Strange! It doesn't happen when I receive just one message- something about two in a row sets it off. Has anyone else experienced this?

tsuehpsyde
Harmless
Posts: 18
Joined: Tue Aug 18, 2009 1:32 am

Re: Prowl notifications for Google Voice

Postby tsuehpsyde » Tue Aug 18, 2009 1:41 am

I've had a few issues (well, *having* a few issues). We'll attack them one at a time. I submitted them to Mike, but he's a busy guy. =) I'm working on trying to diagnose/fix them myself, but I'm fairly novice with Python at this point, but it's always good to learn.

I added some code into this script. Basically, a print time.ctime(time.time()) @ start time, and when running gv.check(), I did try: (command) except: (print time command).

I run my script every 15 seconds, and SMS runs great. Since 2:53PM, I've already had 2 exceptions from the script:

tsuehpsyde@storage:~/bin/prowl-gv$ ./prowlgooglevoice.py
startup: Mon Aug 17 14:53:03 2009
exception: Mon Aug 17 17:52:18 2009
exception: Mon Aug 17 17:56:23 2009

However, it doesn't print any error messages, which is interesting. I've had it completely fault out once before (which I sent the output to MikeK, don't have it on hand). Mainly since I ran the prowl-gv.sh script I wrote and submitted, but would notice randomly it was not running anymore. Weird. I then ran it in a screen and after a day or two I saw a pretty hardcore exception. It appears I get minor ones as well (mind you that during those timeframes, I got no new SMS or VMs). Waiting for it to break so I can post more details, but it appears that something is breaking, I'm guessing on Google's side or possibly on my network here? Not sure.

Second problem: Voicemails don't work when the transcription is not available. I have a 7 second empty VM and I get the following error:

Traceback (most recent call last):
File "./prowlgooglevoice.py", line 66, in <module>
gv.check()
File "/home/tsuehpsyde/bin/prowl-gv2/googlevoicenotify.py", line 139, in check
self.parse_voicemails(vmfeed)
File "/home/tsuehpsyde/bin/prowl-gv2/googlevoicenotify.py", line 117, in parse_voicemails
from_name = message.findAll('a', attrs={'class':'gc-under gc-message-name-link'})[0].string.strip()
IndexError: list index out of range

The actual content of the variable being processed:

Code: Select all

<table class="gc-message-tbl">
<tr>
<td class="gc-message-tbl-portrait">
<div class="gc-message-portrait">
<img alt="Blue_ghost" src="/voice/resources/1366864992-blue_ghost.jpg" />
<div class="gc-message-icon-2"></div>
</div>
</td>
<td class="gc-message-tbl-metadata">
<div>
<span class="gc-message-name">
<span title="">(205) 492-xxxx</span>
<img alt="Speaker" class="gc-message-play-name" src="/voice/resources/539834508-speaker.png" />
</span>
<span class="gc-message-add-contact">
<a href="javascript://">Add</a>
<div class="gc-quickadd-contents" style="display: none;">
<form action="" name="quickcadd" onsubmit="return false;">
<div class="gc-bubble-ih" style="margin-bottom: 3px;">
<span>Quick Add</span>
<span style="font-weight: normal;"></span>
<span style="color: #0f65bb; margin: 0 0 0 2px;">(205) 492-xxxx</span>
</div>
<div class="gc-bubble-subg" style="font-size: 11px; margin-bottom: 3px; white-space: nowrap;">
    Enter a new or existing contact name
  </div>
<table class="gc-quickadd-inputs">
<tr>
<th>Name</th>
<td>
<input type="text" class="gc-text" />
</td>
</tr>
<tr>
<th style="padding-top: 6px;">Type</th>
<td style="padding-top: 5px;">
<div>
<div class="goog-flat-menu-button gc-quickadd-phone-type">
<div class="goog-menu gc-simple-menu gc-phone-type-menu" style="z-index: 10001;">
<div class="goog-option goog-option-selected">Mobile</div>
<div class="goog-option">Work</div>
<div class="goog-option">Home</div>
</div>
</div>
</div>
</td>
</tr>
</table>
<div class="gc-quickadd-buttons">
<div class="goog-inline-block gc-quickadd-save">Create / Add</div>
<div style="display: none;" id="gc-qa-1250198993000" class="goog-inline-block gc-quickadd-msg"></div>
</div>
<div class="gc-quickadd-contacts" style="display: none;"></div>
</form>
</div>
</span>
<span class="gc-message-location">- <a class="gc-under" style="color: #999;" title="Go to map of area code location" href="http://maps.google.com/maps?q=Birmingham, AL" target="_blank">Birmingham, AL</a></span>
</div>
<div class="gc-message-time-row">
<span class="gc-message-time">8/13/09 4:29 PM</span>
<span class="gc-message-relative">4 days ago</span>
</div>
</td>
<td class="gc-message-tbl-actions">
<div class="gc-message-actions">
<div class="goog-inline-block" style="margin-bottom: -4px;"><div style="border-color: rgb(238, 238, 238); background-color: rgb(238, 238, 238);" class="gc-message-label"><div style="border-color: rgb(238, 238, 238);" class="gc-message-label-o"><div style="color: rgb(68, 68, 68);" class="gc-message-label-i">Inbox</div></div></div></div>
<div class="goog-inline-block goog-flat-button gc-message-star gc-message-star-off">&nbsp;</div>
</div>
</td>
</tr>
<tr>
<td colspan="4" class="gc-message-tbl-desc">
<div class="gc-message-transcript">
<div class="gc-message-transcript-msg">
<table class="gc-message-transcript-middle">
<tr>
<td class="gc-message-transcript-l"></td>
<td class="gc-message-transcript-m"><div>
<div class="gc-message-player">
<table class="gc-message-message-tbl">
<tr>
<td>
<div class="gc-message-message-display">
<span class="gc-no-trans">Transcript not available</span>
</div>
</td>
</tr>
</table>
</div>
</div></td>
<td class="gc-message-transcript-r"></td>
</tr>
</table>
</div>
</div>
</td>
</tr>
</table>


However, I did get a VM with a long (49 second) message that sent just fine. I'm currently running SMS-only on this script to avoid the error. Any thoughts would be helpful. :)

atomjack
Harmless
Posts: 6
Joined: Wed Aug 12, 2009 6:03 pm

Re: Prowl notifications for Google Voice

Postby atomjack » Tue Aug 18, 2009 6:25 pm

@tsuehpsyde, your problem with the voicemails not working when the transcription is not available is actually happening because the sender is not in your address book. I played around with it and found that empty voicemails (with no transcription available) work fine, but it was erroring out because it doesn't have a name for the sender, just a phone number. To fix it, replace line 117 in googlevoicenotify.py with:

Code: Select all

            try:
               from_name = message.findAll('a', attrs={'class':'gc-under gc-message-name-link'})[0].string.strip()
            except:
               from_name = message.findAll('span', attrs={'title':''})[0].string.strip()

The line you need to replace is the first 'from_name = message.findAll...' right after the try:. This will just use the phone number as the from name if there's no name associated with the number (in your contacts list).

I'll write Mike soon with this fix, but I need to fix the issue of voicemails appearing twice if the transcription isn't available right away.

If you want to test this with that voicemail, if you haven't deleted it already, just mark it unread in google voice, and remove the .gvcache file from your server.

edit:I've also come across the problem of the script just stopping randomly, either run in a screen session or using the daemon script, so I changed my prowlgooglevoice.py file to not continually repeat - to just run once, and put it in a cron to run every minute. I don't really know what's causing this, and I'm a beginner at python but even so I am not sure that it is a problem with the script. No idea though, really.

tsuehpsyde
Harmless
Posts: 18
Joined: Tue Aug 18, 2009 1:32 am

Re: Prowl notifications for Google Voice

Postby tsuehpsyde » Wed Aug 19, 2009 3:47 am

atomjack,

Thanks! The fix works great. However, I've noticed the text in the voice messages only seems to be relayed if it's classified by the darkest color (the most sure), so a ton of the voicemail message isn't translated. Is there any way around this?

Also, I've setup the script to be a bit more verbose so I can now see what the exception is when it fails. I put the Debug = True up top. Thinking we should probably use this technique instead of commenting out prints.

Code: Select all

Debug = True
<snip>

        while True:
                try:
                        if Debug:
                                print 'checking'
                        gv.check()
                        if Debug:
                                print 'done checking, sleeping'
                        time.sleep(float(sleep_time))
                except (KeyboardInterrupt):
                        print 'Keyboard interrupt detected, exiting.'
                        sys.exit()
                except:
                        print 'Exception:', time.ctime(time.time())
                        raise


I also set the sleep_time to float() as a fix, since it was failing for me saying it wasn't a float. I sent that to Mike as well, which he'll hopefully patch.

Worth noting, if you comment out the raise within the except: portion, it removes the need for putting a shell loop. Any time the gv.check() fails for whatever reason (no idea why it's failing, setting up the raise now to find out), it will just go back to looping and checking (as posted in my previous post).

Great script! Sure glad I have voicemails now. :D Now to just get all the text in there if possible...

Oh, and I added the KeyboardInterrupt, because doing control+c was being caught by except: and I wanted to be able to kill it while testing things.

tsuehpsyde
Harmless
Posts: 18
Joined: Tue Aug 18, 2009 1:32 am

Re: Prowl notifications for Google Voice

Postby tsuehpsyde » Wed Aug 19, 2009 2:54 pm

Here's the exception that's breaking the script intermittently:

Code: Select all

tsuehpsyde@storage:~/bin/prowl-gv$ ./prowlgooglevoice.py
Startup: Tue Aug 18 23:32:56 2009
Exception: Wed Aug 19 07:54:09 2009
Traceback (most recent call last):
  File "./prowlgooglevoice.py", line 71, in <module>
    gv.check()
  File "/home/tsuehpsyde/bin/prowl-gv/googlevoicenotify.py", line 138, in check
    feed = self.get_inbox()
  File "/home/tsuehpsyde/bin/prowl-gv/googlevoicenotify.py", line 66, in get_inbox
    sms = self.do_req('https://www.google.com/voice/inbox/recent/sms/').read()
  File "/usr/lib/python2.5/socket.py", line 291, in read
    data = self._sock.recv(recv_size)
  File "/usr/lib/python2.5/httplib.py", line 509, in read
    return self._read_chunked(amt)
  File "/usr/lib/python2.5/httplib.py", line 554, in _read_chunked
    value += self._safe_read(amt)
  File "/usr/lib/python2.5/httplib.py", line 604, in _safe_read
    raise IncompleteRead(s)
httplib.IncompleteRead:


Then it spews a bunch of XML data (hard to get the exact since I ran in a screen and I can only see it all by using ctrl+u ctrl+d and it doesn't copy right. I'm going to add an except (IncompleteRead) section that simply prints when it happens (like above) so it skips it and keeps going. It seems to happen about 2-3 times a day, on average, fyi.

Setting up to send the exception(s) into a logfile for easier pasting via the traceback module. Will post back if/when I get new stuff.

Edit: Got it setup. log_file is defined as a local filename. Current startup logic for prowlgooglevoice.py:

Code: Select all

    print 'Startup:', time.ctime(time.time())
    while True:
        try:
            if Debug:
                print 'checking'
            gv.check()
            if Debug:
                print 'done checking, sleeping'
            time.sleep(float(sleep_time))
        except (KeyboardInterrupt):
            print 'Keyboard interrupt detected, exiting.'
            sys.exit()
        except (IncompleteRead):
            print 'IncompleteRead:', time.ctime(time.time())
            traceback.print_exc(file=open(log_file,"a"))
        except:
            print 'New Exception:', time.ctime(time.time())
            traceback.print_exc(file=open(log_file,"a"))


Should help out with the errors as well as keep the script running continuously.

Telecart
Harmless
Posts: 2
Joined: Thu Aug 20, 2009 12:02 pm

Re: Prowl notifications for Google Voice

Postby Telecart » Thu Aug 20, 2009 12:29 pm

Woah, this is just what I need...

Any chance someone wraps this up in py2app for osx or writes a n00b-friendly guide on how to set this up?

Not afraid of terminal, but never touched python before...

tsuehpsyde
Harmless
Posts: 18
Joined: Tue Aug 18, 2009 1:32 am

Re: Prowl notifications for Google Voice

Postby tsuehpsyde » Fri Aug 21, 2009 4:01 pm

Okay, so the random crashes are always from an incomplete read from Google. Instead of just visually logging the errors, I decided to fix it within the googlevoicenotify.py library.

The following line in get_inbox() is our offending line:

sms = self.do_req('https://www.google.com/voice/inbox/recent/sms/').read()

So I replaced it with:

Code: Select all

      try:
         sms = self.do_req('https://www.google.com/voice/inbox/recent/sms/').read()
         sp = BeautifulStoneSoup(sms)
         return str(sp.response.html.contents[0])
      # This randomly fails for reasons unknown
      except IncompleteRead, e:
         return None


Code: Select all

      try:
         html = self.do_req('https://www.google.com/voice/inbox/recent/voicemail/').read()
         sp = BeautifulStoneSoup(html)
         return str(sp.response.html.contents[0])
      # This randomly fails for reasons unknown
      except IncompleteRead, e:
         return None


Then to compensate for that:

Code: Select all

   def check(self):
      feed = self.get_inbox()
      if feed != None:
         feed = feed.replace("<![CDATA[", "")
         feed = feed.replace("]]>", "")
         self.parse_result(feed)
         vmfeed = self.get_voicemails()
         if vmfeed != None:
            vmfeed = vmfeed.replace("<![CDATA[", "")
            vmfeed = vmfeed.replace("]]>", "")
            self.parse_voicemails(vmfeed)
            out_fl = open(self.picklefile, 'w')
            pickle.dump(self.convo_threads, out_fl)


So I just have the "New exception" catch on the start of the script to see if anything else breaks, but I propose that fix as a way to keep it from randomly crashing on us. :)

Edit: The same issue happens from get_voicemails:
html = self.do_req('https://www.google.com/voice/inbox/recent/voicemail/').read()

Fixed that one in the same manner.

Edit2: Oh, and make sure you do "from httplib import IncompleteRead" as well, otherwise you can't catch it.

atomjack
Harmless
Posts: 6
Joined: Wed Aug 12, 2009 6:03 pm

Re: Prowl notifications for Google Voice

Postby atomjack » Tue Aug 25, 2009 5:46 pm

Mike updated some of the files in the github with tsuehpsyde's changes, and some I made as well.

My changes are for handling of voicemails when there's no transcript - when it finds one, it skips pushing it the first time, but the next time it goes ahead and pushes it, regardless of whether or not there's a transcript. This way, if it catches the voicemail before google has transcribed it, hopefully the next time it runs there will be one.


Return to “Prowl”

Who is online

Users browsing this forum: No registered users