Thursday, October 21, 2010

Android C2DM Messaging (Server Push for Android Phones)

UPDATE: Added diagrams to get concepts clear.

From monday, I have been fighting to get the C2DM (Cloud to Device Messaging) working. And with somewhat frustration and a lot of help from some good people out there, I got it working ! :)

In this article I describe my findings and try to explain the concepts in simple words and list out steps you need to do to get it working. For an example code I would suggest you take a look at Gabor's blog which is  this. Its really awesome.!

NOTE: Also, I would suggest you to take a look at the sample code provided by google. If I get time, I will also post some sample code (don't wait for me to do that :D). Also, just take a look at the classes in the google.android.c2dm package provided in the sample codes. No need to dig into the details, but I may talk about them later in this post.

So what is C2DM ? It is a server push mechanism provided by Google so that 3rd party applications can push messages to their applications on Android devices. Currently as of today, it is in beta, and if you are an enterprise, you will need to apply and have talks with Google before having your application out with a lot of users and lot of messages to push. However, for developers with a gmail account ID, the registration request should be pretty easily accepted and you will be "whitelisted".

Before digging into details, let us see what are the top level concepts and which components are involved. Basically there are 3 entities involved:
0. The Google C2DM service.
1. Your server application. I'll call it as the "FOOServer". This will send the message to the C2DM service which will forward it to the requested device.
2. The application on the device that receives the message. I'll simply call it DeviceApp.

So, no points to guess that C2DM acts as a middle man and does the message passing. How ? It uses the default google services on the Android phone. So the basic requirement for this whole thing to work is that there should be at least one logged-in Google account from the device.

Now, when you signup for the C2DM, you will be providing them with the following (only 2 relevant things are mentioned):
1. The Application Name. This is actually the package name that you will provide. It is very important to remember this, because this is the package name you will have to use for your DeviceApp and worst part is that in the whitelisted mail that you get from Google, this is not mentioned. I haven't found a way to look at our registered information. If anyone knows this, please comment and let me and other know. So, for the time being consider that the package name is "com.foo.test"

2. The second important thing on the registration form is the Role Account Email ID. Make sure that this email is not used on any device. I have found that the messaging will not work for that device. So, for our example lets say it is FOO@gmail.com

Just for fun, the following diagram depicts the above step.



So, till now, we have the email account ID which is FOO@gmail.com and the package name.

Who needs what
Ok, so how does each entity react to the C2DM and what IDs and credentials does it use ?


DeviceApp
The device app registers with the C2DM service. It tells the C2DM service that it would like to receive messages from FOOServer. In return it gets back a registration ID.
So,
DeviceApp -----[FOO@gmail.com]------> C2DM
C2DM  --------[registrationID]--------> DeviceApp


The first action is done via a registration Intent. If you see the C2DMessaging class you will get this. Here, the DeviceApp sends the FOO@gmail.com address specifying that it would like to receive messages from this account.
Next, You will need to extend the C2DMBaseReceiver and override the onRegistered and onUnregistered methods. So, the onRegistered method will be called when the C2DM server accepts the registration. In this method you will receive a registrationID. This is a very long alpha numeric string that identifies the device.

Now, the DeviceApp needs to convey back this registration ID to the FOOServer. It is recommended that at this point you also send some unique device identifier to keep track on the FOOServer. The reason is that Google may change these registrationIDs and may not be unique. If Google changes any IDs it will push the new ID to the DeviceApp which has to handle it in similar fashion.
So, if you choose say IMEI/MEID as the unique identifier, then,


DeviceApp -----[UniqueID, registrationID]------> FOOServer


Thats it ! The device is now registered and ready to receive any messages. The following diagram shows this registration work flow:


Registration Work Flow





FOOServer:
The FOOServer will store (or update) the registrationID received into its local database. So, eventually the server will have registrationIDs from all the devices the DeviceApp is on. To send a message to a particular device, the FOOServer needs to POST to the C2DM Service the following four three things:
1. The accountName which will be FOO@gmail.com (Update : Thank you @matest for pointing it out that this is not needed)
2. An authentication Token [explained below]
3. The registrationID of the device it wants to send the message.
4. The message itself ( I think the limit is 1024 charcters)


You can get the authentication token by providing the credentials for FOO@gmail.com. I used curl to get the token. In our case the command would be :


curl https://www.google.com/accounts/ClientLogin -d Email=FOO@gmail.com -d "Passwd=FooPassWord" -d accountType=GOOGLE -d source=FOOServer -d service=ac2dm


[note this command is taken from comments posted on this blog]


Getting the auth token




This should give you few very long string values one of which would start with "Auth=". This is the one you want. Even if I have used curl here, it is recommended that you get the token programmability, because even this token might change. If it changes, the new token would be returned as a response to the POST that we do to send the message.

Sending the message



I would like to thank Gabor Paller for writing a very good article on this and helping me with this process. Also, thanks to DavidMyton for his article. The comments below this article are pretty useful too. I also found this discussion on c2dm google group helpful.

I have written this in a hurry. So let me know of any "bugs" in the article and I'll fix those  !!
Thank you !

37 comments:

  1. 1)
    > To send a message to a particular device, the ?
    > FOOServer needs to POST to the C2DM Service the > following 4 things:
    >1. The accountName which will be FOO@gmail.com

    It's not not needed! Only #2,3,4

    2)
    > This should give you few very long string
    > values one of which would start with "authToken="

    not "authToken=", but "Auth="

    ReplyDelete
  2. 3) you should use curl command with -k option if you have some problem with ssl

    ReplyDelete
  3. @matest thanks for the comments, I'v updated the post !

    ReplyDelete
  4. @Oleksandr I didn't get you. Can you please elaborate your question ?

    ReplyDelete
  5. for example: I've got auth token for c2dm service
    (https://www.google.com/accounts/ClientLogin ...)

    When it will be expired? I mean when I need to obtain a new one?

    ReplyDelete
  6. AFAIK, we cannot tell when does it expire. But when it expires, if we make a request with the old token, google will accept the old token and in the response, will give us the new token. It is upto our server to handle this and store the new token. I'm not sure whether google will accept the old token from this point onwards ! HTH

    ReplyDelete
  7. Thank you for the answer Tejas.
    I will implement token update

    ReplyDelete
  8. The token is valid for 24 HRS if there is no attempt made to get a token.

    Another thing I have noticed is that your roleaccount email and the device email can be the same. Messaging works AFAIK.

    ReplyDelete
  9. Hi, maybe this is a funny question but... Do i need a httpS server to implement C2DM?

    ReplyDelete
  10. @Javi, I assume you mean to say httpS "request" to the c2dm servers. If so, yes ! Infact the documentation states to make an https post :
    http://code.google.com/android/c2dm/index.html#push

    ReplyDelete
  11. Thanks for this post Tejas. It seems to touch on all the key ingredients, but I'm struggling with an HTTP 401 (unauthorized) response. Here is the curl request I'm using to send a c2dm push request on behalf of the server:

    curl --header "Authorization: GoogleLogin auth=xxTokenUsingPersonalEmail" "https://android.apis.google.com/c2dm/send" -d registration_id=xxIdFromAppxx -d "data.message=Hello" -d collapse_key=message -k

    The only two variables are the auth= value and the registration_id= value, and I don't think a wrong registration_id will lead to a 401. I generated the auth= value using your curl command line. It's about 45 hours since I registered with Google c2dm service. Not sure if there's a way to find out if I'm fully registered yet or not. though I did get an initial confirmation email.

    ReplyDelete
  12. Hi Tejas...Thanks for the great post...I am having an issue with the code is with I do get registered with the server how ever when I send a message from server back to my device it says message sent successfully however I couldn't be able to get that message displayed on my device...I am using Windows 7 Development environment...the above server side code is running on Google Apps and I also have changed the config file with server dns settings..

    ReplyDelete
  13. @BCC Too much confusing ! I assume that you are able to register your device, then send the registration ID obtained from C2DM to your own server and then you have problems sending a message from your server to the C2DM server. Is that right ? Assuming it is right, When you send message from the server, do you get a confirmation id ? Also, would like to see the logcat from the device. Thanks.

    ReplyDelete
  14. When the first time i run my application...it gives me these errors...

    C2DM Registrar.1071
    Register:http Error 401
    C2DM Reghandle request caught org.apache.http.auth.Authentication Exception

    but on the server end it do display the register user account...and do show me the Registration id...same registration ID which my app in logcat display...and important when i send message it do give me the token as you described...

    I am new to Python so just want to know what are these files for "ae_pushserver.sh" and do I need to upload them as well to Google App?

    Thanks in advance

    ReplyDelete
  15. OK the application is now working what I realize was if you are registered on your cell with the same gmail account as your C2DM gmail account id ...the application won't work...what's happening in the background is...you need at least one gmail main account to utilize all Gmail service on that particular device..so that main GMAIL account can't be the same as your C2DM gmail account...and you also need to check background data and Auto-sync...if you try to remove the main account...it will say something like " This account is required by some applications. You can only remove it by resetting the phone to factory defaults"...so in other words you can't delete your main gmail account on your device without resetting the device...but if you registered your device with other account other then your "C2DM" account it will work...and if you add the C2DM account as a secondary account on your device it will still work...the only thing to remember is make sure your device is not registered with the same C2DM gmail account id as the primary account...on your cell phone...

    ReplyDelete
  16. the above error C2DM Registrar 1071 it will show in your logcat if your Google C2DM service is not running on your device...just to check if the service is running in your android service section....

    ReplyDelete
  17. @Tejas: You can send any gmail id that is synchronized with your phone.
    @Subuk: I need to get the primary gmail id, i already use AccountManager to do that with filtering com.google and it gives all gmail id's that are associated with phone but i am not able to filter it which one is primary even though what i think all the mail id that you enter into a com.google type it enter into a stack and the 0th position mail id is the primary one, i tested this in two phone and it works well but as i know this is not the correct way to do it. Any better way to get primary gmail id.

    ReplyDelete
  18. Hi,

    I am able to successfully implement this and I am able to get notifications on the emulator. Initially the engine took about 2 seconds to send notifications and now, the delay has increased by almost 15 minutes(still getting notifications). I am surprised by this.

    Is there anything in specific that I need to configure ?

    Thanks,
    Vny

    ReplyDelete
  19. Does Google allow the developer to sell the app on android Market if it uses C2DM?

    ReplyDelete
  20. @The Great Mind - Same to me in the beginning 15 min delay. however now im pushing 3-4 messages a day and no delay

    ReplyDelete
  21. @Hans-Erik : Thanks for the reply.

    I donot face any delay if I start sending notifications to a fresh device continuosly. But, If I keep the emulator idle for about 20 minutes and then send a Notification to the emulator, delay is observed then. But still the notification reaches the emulator.

    ReplyDelete
  22. @The Great Mind - There is no way you can configure the timing. While I haven't faced any such delays, Google may send it whenever there is an opportunity. They actually piggy back the message into one of its accounts connections, so there is no guarantee. Even there is no guarantee that the message will ever reach the device. Its upto the Server to determine this and resend the message sometime in future.

    @Hans-Erik - The question about the market is interesting. While C2DM is still in beta, we need to talk with Google if the number of messages we push are greater than what we declared during C2DM sign-up. I have seen some market apps using C2DM !

    ReplyDelete
  23. just got a response from google about using the c2dm.

    "We give everyone 100,000 messages/day and 1,000 messages/device/day to
    start.

    If you expect to need more than this, please let us know. We try to
    respond to all requests within 1 business day. We can't guarantee
    increased quota, but we can usually give it provided that the app in
    question is well behaved. (In short: We want to make sure C2DM is only
    used to wake up sleeping devices, rather than used as a primary
    communication channel.)"

    ReplyDelete
  24. And here goes the finale:

    Erik,
    You're more than welcome to sell your app on Android Market using C2DM.

    The full terms of service are listed here:

    http://code.google.com/android/c2dm/terms.html

    --
    Trevor Johns
    Developer Programs Engineer, Android
    http://developer.android.com

    ReplyDelete
  25. @Hans-Erik Thanks for all the info. I'll put a note in the blog when I get some time.

    ReplyDelete
  26. Hi Tejas,

    How to maintain state less session in android.I need to maintain the session once the customer is logged in.


    Thanks,
    viswanath

    ReplyDelete
  27. @visu Can you please elaborate ? Where do you want to maintain a session?

    ReplyDelete
  28. hi,how to implement local host in windows as you told in Gabor Paller blog.I installed all as there, but not able to have http://localhost:8080. Should i install Tomcat

    ReplyDelete
  29. Did you start the Google App Engine. The App Engine itself serves as a web app. server. So no need of Tomcat.

    ReplyDelete
  30. Dear Tejas,

    I am struggling to implement a server side script or anything to handle the server side of c2dm..I tried to work the example here

    http://mylifewithandroid.blogspot.com/2010/10/push-service-from-google.html

    but with no luck..can you upload an example of the server side app/code?

    ReplyDelete
  31. hello....
    how i make a chat application using C2DM....
    if 1 device sends the message then it will be push to the another device and vice-versa.

    plz help me...!!!

    ReplyDelete
  32. Hello

    I will create an app. using C2DM service

    according to the relevant materials Quotas 200,000 messages per day are taken to the base.

    in 200000 messages , registration_id, Auth Token is included calls wondering about?

    I want to know how to change registration_id value and How to know randomly changes the value before the change to know that there any?

    plz help me~~~

    ReplyDelete
  33. hi all
    I not recived mail sinced singup c2dm more 5 day? Why

    ReplyDelete
  34. hello Trọng LÊ,

    C2DM service is deprecated
    you shold use GSM service instead
    check link : http://developer.android.com/guide/google/gcm/gs.html

    ReplyDelete