406 replies to this topic

[shenor]

If this help, this my hac log

625]2012-12-30 19:46:03 +0200 Messages[28454]: [Warning] Empty account query with service: iMessage

625]2012-12-30 19:46:03 +0200 com.apple.SecurityServer[15]: Session 100068 created

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] *** Listener ID: com.apple.mail  does not have capability: (Status), not allowing request

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 1   IMFoundation                        0x00007fff82fdcbe2 IMLogBacktraceToDepth + 69

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 2   imagent                             0x000000010c94ade8 imagent + 81384

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 3   imagent                             0x000000010c94bf6a imagent + 85866

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 4   CoreFoundation                      0x00007fff8dcc463c __invoking___ + 140

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 5   CoreFoundation                      0x00007fff8dcc44d7 -[NSInvocation invoke] + 263

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 6   CoreFoundation                      0x00007fff8dcc46a9 -[NSInvocation invokeWithTarget:] + 57

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 7   IMFoundation                        0x00007fff82fd95d5 im_local_object_peer_event_handler + 7128

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 8   IMFoundation                        0x00007fff82fd96b3 im_local_object_peer_event_handler + 7350

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 9   IMFoundation                        0x00007fff82fd9767 im_local_object_peer_event_handler + 7530

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 10  Foundation                          0x00007fff8ed40677 __NSThreadPerformPerform + 225

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 11  CoreFoundation                      0x00007fff8dc50101 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 12  CoreFoundation                      0x00007fff8dc4faed __CFRunLoopDoSources0 + 445

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 13  CoreFoundation                      0x00007fff8dc72dc5 __CFRunLoopRun + 789

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 14  CoreFoundation                      0x00007fff8dc726b2 CFRunLoopRunSpecific + 290

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 15  Foundation                          0x00007fff8ed4889e -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 268

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 16  Foundation                          0x00007fff8ece118a -[NSRunLoop(NSRunLoop) run] + 74

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 17  imagent                             0x000000010c93f0ab imagent + 32939

625]2012-12-30 19:46:20 +0200 imagent[197]: [Warning] 18  libdyld.dylib                       0x00007fff854fb7e1 start + 0

[lisai9093]

rcork, on 30 December 2012 - 04:29 PM, said:

These log entries are from my hackintosh. However, i was curious if these log entries showed up on my "real" macbook pro and indeed they do. So these errors are not specific to just my hackintosh.

I took this approach, too. But bad news is that I found no evidence that Mac had the same log entries as hackintosh's. I'm wondering how apple distinguish a Mac and hackintosh. If we know it, we might crack it.

[eep357]

As I mentioned before, it's failing when trying to get a valid token from apple server and the handshake process is detailed here http://imfreedom.org/wiki/IMessage which also has directions to enable advanced debugging for applepushserviced which appears to be where the failure is. There are 3 seperate process involved in making the connection/validation: iChat, IMAgent and applepushserviced. Using a network monitor (i.e. Hands Off!) you can see what connections each process makes and data sent/recieved. Doing the same on a real Mac for comparison would be super helpful! There is also a link for github project with tools for debugging and setting up a push proxy (but this is more work around than actual fix) I need to finish fine tuning new HD7970 and watch Football before I can look into it too much more, and only real mac I have won't run ML. Last week before leaving for holiday quickly copied network info from IMagent but not the push service as I should have done, but here is what I'm talking about:

Connection history for IMRemoteURLConnectionAgent (/System/Library/PrivateFrameworks/IMFoundation.framework/XPCServices/IMRemoteURLConnectionAgent.xpc/Contents/MacOS/IMRemoteURLConnectionAgent)Total sent: 56.30kB Total received: 85.69kB service2.ess.apple.com (17.154.239.37) on port 443 (https) using protocol 6 (TCP), sent: 22.01kB and received: 12.63kBstatic.ess.apple.com (216.156.147.11) on port 80 (http) using protocol 6 (TCP), sent: 448B and received: 14.46kB service2.ess.apple.com (17.173.255.73) on port 443 (https) using protocol 6 (TCP), sent: 22.02kB and received: 12.62kB static.ess.apple.com (140.174.24.81) on port 80 (http) using protocol 6 (TCP), sent: 448B and received: 14.46kB service.ess.apple.com (17.173.255.51) on port 443 (https) using protocol 6 (TCP), sent: 5.20kB and received: 6.60kB init.ess.apple.com (216.156.147.57) on port 80 (http) using protocol 6 (TCP), sent: 112B and received: 4.23kB service.ess.apple.com (17.173.255.24) on port 443 (https) using protocol 6 (TCP), sent: 5.75kB and received: 7.99kB init.ess.apple.com (140.174.24.32) on port 80 (http) using protocol 6 (TCP), sent: 224B and received: 8.46kB init.ess.apple.com (140.174.24.25) on port 80 (http) using protocol 6 (TCP), sent: 112B and received: 4.23kB

I would expect only the https connections to be involved in authentication. Need to see exactly which server connection is the failure point and if it's the sent or received response that triggers it, you can then look on the web page I linked and see in the examples what info is sent to, or received from that server during that step of the handshake. Again same info from real mac needed too. Then we can have it narrowed down to the actual cause and stop making wild ass guesses but instead actually start working toward a solution!

P.S: I think it's fair to say that without further debugging enabled, the same console logs being posted everywhere are useless.

[lisai9093]

rcork, on 30 December 2012 - 04:29 PM, said:

These log entries are from my hackintosh. However, i was curious if these log entries showed up on my "real" macbook pro and indeed they do. So these errors are not specific to just my hackintosh.

That's a good news! But I'm wondering why all hackintoshs are down but not so many macs.

[p.H]

lisai9093, on 30 December 2012 - 11:15 PM, said:

That's a good news! But I'm wondering why all hackintoshs are down but not so many macs.

I once made a guess. Due to heavy load on Apple imessage server, Apple has to restrict the usage of Message. They may change the rules they used to accept the verification. Anyway, just personal opinion

[eep357]

p.H, on 31 December 2012 - 02:32 AM, said:

I once made a guess. Due to heavy load on Apple imessage server, Apple has to restrict the usage of Message. They may change the rules they used to accept the verification. Anyway, just personal opinion

You make a good point, it is a free service after all so I would understand them not wanting to upgrade so Hack's can use it and may be same reason Lion beta was discontinued. Similar to how unsupported devices were overloading the Siri servers, otherwise they probably wouldn't have made such effort to block it. iMessage was getting slow to send and receive at times, same as how Siri timeouts would occur when Spire came out.

[fffeee]

This issue also affects virtualized OS X systems, and those are a supported configuration by Apple, so I am inclined to believe that this is not intentional.

There seem to be some commonality with tokenized logins for other services, and authenticating those as well.  Case in point, I changed the password on my iCloud account that I use with iMessage, since I wanted to force my tokens to refresh on my devices.  Long story short, I can't login to iCloud.com web services with my new password, nor can I use iMessage on my Hacs.  I'll see if my phone and iPads are punted as expected but if they don't let me re-authenticate either I believe this is an accident or symptomatic of a larger issue.

[fffeee]

an addition worthy of not editting my previous post:

$ curl --data-ascii @f  https://service.ess.apple.com/WebObjects/VCProfileService.woa/wa/authenticateUser  
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>message</key><string>status = 5012, INVALID NAME OR PASSWORD</string>
  <key>status</key><integer>5012</integer>
</dict>
</plist>

Presently this is the result I'm getting from the imessage auth service, where @f is of course the file containing my plist with my authentication bits.  If someone else that knows about the iMessage auth service and associated bits, please let me know if you're getting the same message without a recent password change and we'll see if something is happening with the iMessage auth service in general, or if this is because I changed my password and it hasn't propogated to that service yet (which doesn't make sense seeing as how I think they're using kerb anyway).

Here's an example you can use for what you should be POSTing to that url.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	   <key>password</key>
	   <string>lkjasdkfjlk;ajsdf;ljasdfj</string>
	   <key>username</key>
	   <string>myappleid@me.com</string>
</dict>
</plist>

[STLVNUB]

@fffeee
And the plist that comes back with the authorisation
What is it supposed to be called, and where does it go??

[fffeee]

It will contain an auth token in the plist, and you use that token when you generate a PEM-encoded CSR to send back to the service.  This isn't going to fix your iMessage in one step, I'm trying to see where it is failing.

When iMessage talks to various web services it prepends some headers to the POST like:

x-ds-client-id: t:(hexadecimal string to identify software or computer/device)
x-protocol-version: 4
x-vc-profile-id: D:(the profile-id)
x-vc-auth-token: (the auth token)

So, for whatever reason, it seems likely that the ds-client-id that is generated by your virtual machine or Hac is probably failing, and it isn't widely known what the method used for generating that is. Or, some people have mentioned they have logs showing a failure here when it is successful in logging in.

[Adam1203]

I noticed that my x-ds-client-id is null....

I have a feeling that a new FakeSMC will be needed.

[rcork]

fffeee, on 31 December 2012 - 01:56 PM, said:

It will contain an auth token in the plist, and you use that token when you generate a PEM-encoded CSR to send back to the service.  This isn't going to fix your iMessage in one step, I'm trying to see where it is failing.

When iMessage talks to various web services it prepends some headers to the POST like:

x-ds-client-id: t:(hexadecimal string to identify software or computer/device)
x-protocol-version: 4
x-vc-profile-id: D:(the profile-id)
x-vc-auth-token: (the auth token)

So, for whatever reason, it seems likely that the ds-client-id that is generated by your virtual machine or Hac is probably failing, and it isn't widely known what the method used for generating that is. Or, some people have mentioned they have logs showing a failure here when it is successful in logging in.

when i execute the curl command you posted above i receive the xml formatted plist with the auth-token, profile-id and status keys. How can i check for the x-de-client-id?

[iFIRE]

I have the problem, FaceTime and App Store iTunes all is ok, but Messages not working


bash-3.2# curl --data-ascii @f  https://service.ess....uthenticateUser
Warning: Couldn't read data from file "f", this makes an empty POST.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com...yList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>message</key><string>status = 5008, missing required key: username</string>
  <key>status</key><integer>5008</integer>
</dict>
</plist>
bash-3.2#

[fffeee]

rcork, on 31 December 2012 - 05:35 PM, said:

when i execute the curl command you posted above i receive the xml formatted plist with the auth-token, profile-id and status keys. How can i check for the x-de-client-id?

Your results are confirming that the issue lies entirely with the x-ds-client-id.  Thank you for taking the time to test.  As for how to check for the x-de-client-id, that's where we're going to be stuck.  We don't even know why the one it was sending previously isn't valid after the Dec 18th maintenance that Apple did, or if this is intentional or an outage condition they are going to resolve.

The good news is that you're able to authenticate successfully and that the problem isn't at the account level. It is likely that the client-id is being calculated in a different way for some reason, but without further work I don't know that.  If I can spend time today in layer 4 I'll do so and post my findings.

[rcork]

iFIRE, on 31 December 2012 - 05:57 PM, said:

I have the problem, FaceTime and App Store iTunes all is ok, but Messages not working


bash-3.2# curl --data-ascii @f  https://service.ess....uthenticateUser
Warning: Couldn't read data from file "f", this makes an empty POST.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com...yList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>message</key><string>status = 5008, missing required key: username</string>
  <key>status</key><integer>5008</integer>
</dict>
</plist>
bash-3.2#

Change @f to @[name of plist]. For example, i copied the xml from @fffeee's post into a file named imsg.plist. Then the command is curl --data-asci @imsg.plist .....

[fffeee]

iFIRE, on 31 December 2012 - 05:57 PM, said:

…
Warning: Couldn't read data from file "f", this makes an empty POST.
…
bash-3.2#

You aren't doing it right.  The file you reference needs to contain a plist with your credentials. If you aren't clear what you're doing, it's fine, you don't need to help and what you get back will not get you on iMessage anyway, it's just to confirm where the issue lies.

And for the love of god, don't run random curl commands as root

One thing that may be interesting is that I can get iMessage on my Hac to enumerate the email addresses associated with the account.  That doesn't appear to require a valid x-ds-client-id though, but does explain why I can successfully login to iCloud on my Hac, and then initiate the login to iMessage and see it list my email addresses before it fails to register.  Getting a list of email addresses for an AppleID doesn't require that string.

[eep357]

I'm also seeing the push service making a connection too x.courier.sandbox.push.apple.com which I've not seen before. Connection to x.courier.push.apple.com is part of token/auth but what's up with the sandbox one? Is same seen on real Mac?
--------------------------------------------------------------------------
what needs to be done, preferably on real Mac first: after patching applepushserviced, editing hosts file and extracting a valid cert from keychain is seting up a middle-man proxy server which requires redirecting push connections to your server and setting up X.509 certificates for SSL authentication.
The push protocol uses SSL for client and server authentication. You need to extract the valid certificate and give it to the proxy so it can authenticate itself as valid Mac against Apple. You also need to make the Mac trust your proxy since you are impersonating Apple's servers.
The above info and tools to create proxy and extract certs, edit host file etc. is all at https://github.com/meeee/pushproxy. It's primarily focused on iOS but includes everything for OSX too.

[fffeee]

eep357, on 31 December 2012 - 07:21 PM, said:

I'm also seeing the push service making a connection too x.courier.sandbox.push.apple.com which I've not seen before. Connection to x.courier.push.apple.com is part of token/auth but what's up with the sandbox one? Is same seen on real Mac?

That happens all the time; APN can be used by just about anything and doesn't appear to be related to the iMessage registration or initialization process. Yes it will probably use your Token and yes, it probably has your AppleID (it's just XMPP traffic after all) but I don't think it's involved in our iMessage problem.

I have mitm'ed successful iMessage registrations from my rMacBookPro10,1 and a MacMini5,3 and don't have a Hac available until I get home to compare, but I'll be able to see what my Hac is sending for a client-id if anything and if it's even resembling what my Macs send.

I don't know where to begin figuring out how it builds an x-ds-client-id that is way out of my league.  I'm not willing to share my x-ds-client-ids from my Macs either since they're apparently uniquely identifying in some way.  Worst case scenario is using a proxy to intercept and replace bad data with good data, but that is too fiddly and I'd rather just not have iMessage on my Hac than have to deal with that nonsense.

FWIW I'm using mitmproxy to observe/record and play back my iMessage sessions.  Talking about how to do that is out of scope for this conversation but if you're comfortable with tools like tcpdump and can install python modules you can probably get it rolling on your own.

[eep357]

@fffeee: aren't virtualized OSX's only Apple supported when running from Mac which is running OSX server?

fffeee, on 31 December 2012 - 07:41 PM, said:

That happens all the time; APN can be used by just about anything and doesn't appear to be related to the iMessage registration or initialization process. Yes it will probably use your Token and yes, it probably has your AppleID (it's just XMPP traffic after all) but I don't think it's involved in our iMessage problem.

I have mitm'ed successful iMessage registrations from my rMacBookPro10,1 and a MacMini5,3 and don't have a Hac available until I get home to compare, but I'll be able to see what my Hac is sending for a client-id if anything and if it's even resembling what my Macs send.

I don't know where to begin figuring out how it builds an x-ds-client-id that is way out of my league.  I'm not willing to share my x-ds-client-ids from my Macs either since they're apparently uniquely identifying in some way.  Worst case scenario is using a proxy to intercept and replace bad data with good data, but that is too fiddly and I'd rather just not have iMessage on my Hac than have to deal with that nonsense.

FWIW I'm using mitmproxy to observe/record and play back my iMessage sessions.  Talking about how to do that is out of scope for this conversation but if you're comfortable with tools like tcpdump and can install python modules you can probably get it rolling on your own.

I think an intercept proxy is needed, at least until enough info is gathered on x-ds-client-ids. With all this and new experimental GPU, I'm officially back in testing mode which makes having my install on RAID-0 a huge PITA(for example can't boot with -x or -f) and non-SSD so slow I want to cry, ergo I will break my array, update sandforce firmware while I have the chance, and do two fresh SSD install's to begin experimenting with.

[fffeee]

If I were in your position I'd not sweat iMessage, you have enough on your plate   I'm dealing with some new storage shenanigans myself.  

I did watch iMessage in dtrace enough to see that it does write a 40 byte value into keychain at least temporarily, which may be the client-id, but I can't find any occurrence of the values of mine in the keychain files afterwards.  

Also I noticed that before it writes 40 bytes of data to keychain, it generates a series of random data using DevRandomGenerator.  16bytes, then 8bytes, then 8bytes, and a final 8bytes.  Could that be where the 40byte client-id is generated?  Maybe!  Is it actually making a random string for that?  Maybe!  More likely is that it is using some sort of cipher against a known value to calculate those bits, I suspect.  May be helpful to someone that it's 16, 8, 8, and 8 though.

eep357, on 31 December 2012 - 08:04 PM, said:

@fffeee: aren't virtualized OSX's only Apple supported when running from Mac which is running OSX server?

Yes, but those don't work either.  (Confirmed on my MacMini5,2)

eep357, on 31 December 2012 - 07:21 PM, said:

what needs to be done, preferably on real Mac first: after patching applepushserviced, editing hosts file and extracting a valid cert from keychain is seting up a middle-man proxy server which requires redirecting push connections to your server and setting up X.509 certificates for SSL authentication.
…

FWIW I think this is the wrong path to take.  I don't believe APN is involved in this at all, my use of dtrace and mitmproxy to hijack the https sessions that iMessage/imagent makes doesn't give me any reason to believe APN is even used by registration or initialization. I'm not an expert on undocumented APIs or anything, though. I don't intend to discourage someone from attempting it anyway, but I don't know how that will help anyone figure out how a client-id is being generated, and that is absolutely where this process is breaking down.

Sorry for my ninja-edits on my last post, I didn't realize it was re-quoting into my previous post.