YMSG protocol support

[crashdemons]

NOTICE: As Yahoo has terminated Yahoo Chat, I am no longer maintaining this code. Thanks.

This particular script provides many, many functions and definitions for working with Yahoo Cookies, Pager, Chat, CAPTCHA, etc.

Current Version: YMSGLib 3.4

NOTE: Do not ask me for help with "Booters", Flooders, Bots, or anything to abuse other users.

License:

YMSGLib.au3 and the small include file for WinHTTP, WinHTTP_COM.au3, are licensed under WTFPL version 2 (alternative link), a GPL-compatible and FSF-approved license. The entire license can be viewed in either source file. If the 'wording' is not acceptable to this forum, I will gladly write a license just for this purpose.

Downloads:

YMSGLib.au3

WinHTTP_COM.au3

Optional Downloads:

YMSGLib_INI.zip

YMSGLib_Examples.zip

Currently included Examples:

Cookie Decoder (some results need verification)

Simple Login and Console output for buddylist/packet data.

Simple Chat Client (Pager Login/PM/Captcha/Chat/Roomlist support)

*Simple Login/PM Pager Client (and HTTP Proxy example copy)

*Simple Roomlist example

Account Existence checker

(* = Requested Script)

Technical Requirements:

Base64 Support:

Base64.au3 (Author: Ward) (Needed for: Cookies, Auth16)

(or alternatively _Base64.au3 by Mikeytown2/blindwig)

MD5 Support:

Crypt.au3 (Author: monoceres)

(or alternatively MD5.au3 by Ward) (Needed for: Auth16)

(or alternatively Hash#[DLL].au3 by Ward)

(or alternatively MD5.au3 by SvenP/Frez Systems Ltd.)

WinHTTP Support:

WinHTTPRequest Object (Needed for: Token Login/ValidateFields Support)

(WinHTTP_COM.au3 as supplied above)

Data Object Support:

ADODB.Stream Object (Needed for: Reading of WinHTTP Data)

YMSGLib Updates:

2.1 - Validation/Status Check functions; LibRequire edit

2.2 - fixed a bug in _ValidateID

2.3 - moved textual information to YMSGLib.ini, Changed Calls to Executes, added (Packet|Status|Field|List)GetName functions.

2.4 - Added Visibility and Away State packet functions, added some documentation

2.5 - VerifyContact fixed; applicable pager packets updated with the destination-network parameter. ($iService)

N/A - Small Include Filename Switcheroo. (Minorly Breaking)

2.6 - Information about the two bytes following the "ver" has been revealed - some ability has been added here, but this is also a script-breaking change.

2.7 - Many functions requiring HTTP now do not rely on WinHTTP; added some replacements to _YMSG_StripFormatting.

2.8 - Fixed a bug in HTTP transfers concerning binary data

2.9 - Added 3 types of ping packets

3.0 - Base32-int conversions and internal int-string functions (usable for Y64-ints)

3.1 - Mobile / SMS functions (thanks to Lost_Protocol, WickedCoder for information)

3.2 - URL arguments for PwToken and ValidateFields functions are now properly escaped.

3.3 - Additions and changes from observation of Yahoo Messenger 10 (see post)

3.4 - Captcha modifications

Works Cited:

Yahoo SMS Specification

LibYahoo2 - YMSG9

The YahELite chat client. (through observation and analysis of activity)

Observation and review of the source code for various chat clients (Gaim [now Pidgin], etc.)

In-Depth Analysis of .... 6 Jan 2008. XSSed.com (Cookie Info)

A Post-mortem of Yahoo! Account Security. (Cookie Info etc.)

(More official source citations not available as YMSG is not an officially published protocol; some sources posted in subsequent replies)

To Do:

N - Rewrite MD5 and B64 functions to look decent (perhaps array/loop)

T - Update the AuthResponse functions to allow status' other than 12 (Invisible)

H - Update example scripts for compatibility

H - Document all YMSGLib function return values

H - Add Away-State and Visibility support to the Chat-Client example

E - Verify the timestamp integer values in the cookie decoder

H - Add full hierarchy processing for the buddylist structure.

H - Change out the Chat Client Example's buddylist with something decent.

E - Add BuddyIcon/Avatar upload support to YMSGLib

B - Make all functions work perfectly with the new HTTP transfer functions (B; some functions require HTTPS)

P - Add accept/deny buddy responses to YMSGLib and processing to the Chat Client

P - Add cross-service support to VerifyContact, BuddyAdded, etc.

* (B=Blocked by an issue/not possible, P=Pending testing, H=potential Headaches, T=subject to Time constraints, E=needs a working Example/documentation, N=None)

/me falls asleep on keyboard

Edit: Corrected written version number in post

Edit: Corrected unterminated strikeout text

Edited April 8, 2013 by crashdemons

[crashdemons]

Here is a screenshot of the cheaply made test-client in Yahoo's Computers Lobby:1

- At least it gets into chat!

I did not spend alot of time developing the client, as the purpose was to simply make an example for YMSG functions (NOT to be pretty or have alot of utility)

Screenshot

Here is a screenshot of the cookie decoder (sorry for all the marked-out information, just protecting my account)

Screenshot

Edited May 7, 2009 by crashdemons

[crashdemons]

Updated YMSGLib and the Chat Client Example

YMSGLib (v1.3c) now supports field counts, field reading by occurrence and corrected a typo in the YMSG_Client_PM function.

The Chat Client now supports a userlist, PM windows, and strips fonts and formatting out.

(until I decide that I want an HTML/RTF screen instead of a textbox)

The Join window now has options to Join/Leave/End Chat (roomlist selector will be added in a future update)

Additionally there is now a Chat Room button under the userlist - you can click it to start chat, once chat is started it serves as a quick way to bring up the Join/Leave/End window - allowing you to quickly switch rooms or leave the current room, or stop chat.

I have updated the above screenshot of the client

Edit: Fixed missing Open PM button in example. (not shown in SS)

Edited May 7, 2009 by crashdemons

[Andreik]

Nice work. I tried some time ago what you already done. Where you find documentation about this?

EDIT: Your client is a little too big to be easy to read. Could you make a lite version, just with some features like:

- a simple GUI to login

- a simple GUI to can press a button to send a PM (without chatroom)

This should be a nice start for everyone. I don't have so much time to read all your code but a lite version should be very interesting. From me 5 starts.

Edited May 7, 2009 by Andreik

[crashdemons]

Nice work. I tried some time ago what you already done. Where you find documentation about this?

I have been using Yahoo chat for a long time - when I started using the YahELite chat client I studied the output of the packet debug window compared to packet logs I had collected via sniffer - later I investigated the source of similar clients and libraries on the internet.

However, most of the open information was somewhat outdated, but still valid when taken with a grain of salt.

Eventually I had well collected information on the packet structure, common packets, common field #'s and their values.

Not to mention that the fact that YahELite Beta versions had a /pkt command allowing you to input custom packets as cleartext and you could use the Debug window to watch what was outputted - this also moved along the comprehension of what was going on.

Since documentation is openly limited on YMSG, I would suggest looking everywhere, including source code websites and everything else.

Links:

YMSG9

Notes that may be helpful to you:

ylogin.txt

YMSG16.TXT

Yahoo Error codes list (as taken from the registration server)

Yahoo Room Category-List (en-us)

Yahoo Category 1001 Room-List (en-us)

Could you make a lite version, just with some features like:

- a simple GUI to login

- a simple GUI to can press a button to send a PM (without chatroom)

The below script is as simple as I can make it.

For the sake of simplicity, all PM's are sent and received in the same window.

Y_PM_Example.au3 is now in the Example ZIP (first post)

I would love to login with one function call, but that's not how the YMSG16 login works.

(AuthRequest Out, AuthChallenge In, AuthResponse Out, AccountInfo In - if the login all goes well)

Edited May 8, 2009 by crashdemons

[crashdemons]

Update: YMSGLib 1.5c

These changes to YMSGLib should not cause any compatibility breaks with your current programs.

Added two functions:

_YMSG_PwToken_Get(ByRef $iError, ByRef $sPwToken, ByRef $sPartnerID, $sUser,$sPass,$sChallenge='')
_YMSG_PwToken_Login(ByRef $iError,ByRef $sCrumb,ByRef $sYCookie, ByRef $sTCookie, Byref $sExpire, $sPwToken)

These functions were added so that your programs can request this information easily and separately.

The _YMSG_Auth16() function now uses the two above functions for simplicity.

Three internal functions were added to allow different MD5 and Base64 Libraries to be used:

__YMSG_MD5bin($str)
__YMSG_B64EncodeStr($str)
__YMSG_B64DecodeBin($str)

Because of the above added functions, the Auth16 function supports two MD5 Libraries and the Y64 function supports two Base64 libraries.

You may now use any of these libraries for MD5:

MD5.au3 (preferred)
                  Author: Ward
                  Function: _MD5($Data)
                  Return: Binary
             MD5.au3
                  Author: SvenP / Phil Fresle / Frez Systems Ltd.
                  Function: MD5($sMessage)
                  Return: Hex String

You may now use any of these libraries for Base64

Base64.au3 (preferred)
                  Author: Ward
                  Function: _Base64Encode($Data, $LineBreak = 76)
                  Function: _Base64Decode($Data)
                  Return: String
             _Base64.au3
                  Author: blindwig / Mikeytown2
                  Function: _Base64Encode($s_Input, $b_WordWrap='', $s_ProgressTitle ='')
                  Function: _Base64Decode($s_CypherText, $s_ProgressTitle = '')
                  Return: String

Additionally, if one of these functions are not present when required, you will see a critical error messages before your program exits.

Update: Cookie Decoder Example

The cookie decoder has been shortened because of the two new PwToken functions added to YMSGLib.

The cookie decode now also requires YMSGLib v1.5c or higher.

Sidenote: the WinHTTP.au3 file included has also been updated

If the WinHTTP/ADODB/Error objects are already initialized, it will not attempt to re-assign them.

The Login/PM example originally posted above is now in the Examples ZIP.

Edited May 8, 2009 by crashdemons

[Andreik]

This was a hard work to study outputs from others yahoo clients but with great results. Thanks for sharing.

[crashdemons]

Update: YMSGLib v1.6c

Added functions for Yahoo Room-List support

Added a version requirement function that your client programs can use in the future to specify which version of YMSGLib is required (either "exactly X" or "equal to or greater than X")

NOTE: At this time hierarchy detection for the room-list categories is not available.

Addition: Room List GUI Example

This example shows the categories and allows you to select them to display the contained lobbies, as well as select specific lobbies.

Update: Chat Client Example

The Join window now has a button ["..."] beside the room input box which will allow you to open the room-list shown in the above addition, but when you select a lobby the name will return to the Join Room input box, allowing you to find and join a room easily.

[Andreik]

Is there any way, after connecting to yahoo to get your friends list, to not be force to write the ID everytime for user who want to PM?

[crashdemons]

Is there any way, after connecting to yahoo to get your friends list, to not be force to write the ID everytime for user who want to PM?

Yes, when you first log into yahoo, Yahoo transmits your remote buddylist and ignore list.

You should (and I need to check this) be able to read them in much the same way that I read the room's userlist - a list of fields.

It is conceivable that I could create a GUI with a ListBox containing the usernames on your buddylist so that you can quickly open a PM/add/remove them.

Of course - this will make the simple examples a bit longer.

- I am also surprised more people haven't openly taken interest in this project, seeing as they can create their own Yahoo clients with what is shown here in the examples.

Edit: Accidentally typed Roomlist instead of Userlist in this post

Edited May 11, 2009 by crashdemons

[Merovingian]

- I am also surprised more people haven't openly taken interest in this project, seeing as they can create their own Yahoo clients with what is shown here in the examples.

Chances are people commonly don't know what "YMSG Protocol" is...as in the thread tittle. Where "Yahoo! Messenger Client Example in AutoIt" might draw more attention?

<CLINK> <CLINK> Two cents deposited. Rawk ON! MOAR Y! PØWER!

[EDIT]: whøøps, "YMSG protocol support, Yahoo Chat, Cookie reading, etc."

All I noticed was Thread tittle as: YMSG Protocol Support on the Thread Header.

Edited May 10, 2009 by Gargy

[ThuongTin]

C:\Documents and Settings\Thuong Tin\My Documents\YM\YMSGLib.au3(111,33) : ERROR: _Hash(): undefined function.

$sHash=Call('_Hash','MD5',$str)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\Thuong Tin\My Documents\YM\YMSGLib.au3(113,26) : ERROR: MD5(): undefined function.

$sHash=Call('MD5',$str)

[ThuongTin]

i have problem with your example, data received from Yahoo has been showing duplicate

[crashdemons]

@Undefined functions -

- I can not think of any other way to conditionally check for a function's existance

- so using a call to a function that does not end the script when an error is found is essential. - that error message is intended to occur. - try disabling Au3Check.

- To my knowledge, Call() does not create any critical errors that end execution when there is a problem.

- The other option is to use Execute() in a similar manner, but that may create higher inefficiency.

@Double PM's -

- Depending upon what server you and a buddy are on, you may receive a secondary message. (Yahoo DOES actually send that duplicate.)

I will look at ways of discarding the secondary message when I update the client.

[crashdemons]

After some studying of what was going on I noticed that the duplicate/resent PM's received have a 450 field with the value "1", while non-resent PM's either contain "0" or do not have this field at all.

I have updated the Chat Client example and the PM example to discard PM's with this field set to 1.

(hopefully this removes unintentional duplicates for the vast majority)

I have also added a limited buddylist to the Chat Client example. (click [Pager] to the left of that chat input box)

(Not added to the PM Example - it took long enough to debug once!)

YMSGLib 1.9c

Added some pager packets.

Changed some packet processing (non-breaking)

Added a packet length function so that the length can be retrieved before processing.

^ Because of some of the above changes, the Chat Client and PM Example now have some better checking of whether a partial packet was received.

PS:

I know the buddylist isn't EXACTLY what you hoped for - but it's a bit difficult to save the hierarchy of all of the buddy groups - maybe that will be added later.

You can double-click on the buddylist to open a PM (or on ignorelist to un-ignore a user)

You can now double-click in the userlist of the Chat Client to open a PM also.

/me passes out from exhaustion.

Edited May 11, 2009 by crashdemons

[ThuongTin]

Duplicate (resent) PM droppedthuongtin88 <To: yh_vnn>: Yeah uki

Duplicate (resent) PM dropped

Thanks you!

[Andreik]

Yes, when you first log into yahoo, Yahoo transmits your remote buddylist and ignore list.

You should (and I need to check this) be able to read them in much the same way that I read the room's userlist - a list of fields.

It is conceivable that I could create a GUI with a ListBox containing the usernames on your buddylist so that you can quickly open a PM/add/remove them.

Of course - this will make the simple examples a bit longer.

- I am also surprised more people haven't openly taken interest in this project, seeing as they can create their own Yahoo clients with what is shown here in the examples.

Edit: Accidentally typed Roomlist instead of Userlist in this post

I should be able to can read buddylist and ignore list but I don't know when yahoo server send this from my client. You write YMSGLib, you should check where this thing happen. A little example would be great.

[crashdemons]

I should be able to can read buddylist and ignore list but I don't know when yahoo server send this from my client. You write YMSGLib, you should check where this thing happen. A little example would be great.

The Buddylist and IgnoreList occurs in packet 00F1, where fields 302 and 303 mark the beginning and ending of a list (which one chosen by the number in these field's data) and fields 300 and 301 mark the beginning and end of list items.

These are arranged so that if the field order is 302,300,7,301,303 then the field 7 is in this particular item which is also in a list.

The hierarchy of the real buddylist/etc. is somewhat more complex:

List:BuddyGroups( Item:BuddyGroup( Field:GroupName & List:Buddies( Item:Buddy(Field:BuddyName), ... ) ), ... )
List:Ignores(Item:Ignore(Field:BuddyName), ... )
List:BuddyStats(Item:BuddyStat(Field:BuddyName & Field:AwayState &? Field:AwayText), ... )

So pretty much the idea is that Lists contain Items, Items contain Fields and/or Lists.

Here is an example that outputs the buddy/ignore information to the SciTE console in correct hierarchy.

(Note: " Case '00F1', '00F0' " is where the fun starts )

-Attachment Removed, see Examples in first post-

(Note: you'll have to close it via the tray icon)

Example output:

StartList(BuddyGroups) {
    StartItem(BuddyGroup) {
        GroupName(SomeGroupNameHere)
        StartList(Buddies) {
            StartItem(Buddy) {
                BuddyName(UserXXX3823)
            } EndItem(Buddy)
            StartItem(Buddy) {
                BuddyName(UserXXX0725)
            } EndItem(Buddy)
        } EndList(Buddies)
    } EndItem(BuddyGroup)
    StartItem(BuddyGroup) {
        GroupName(SomeOtherGroup)
        StartList(Buddies) {
            StartItem(Buddy) {
                BuddyName(Bob349fc_76fsd)
            } EndItem(Buddy)
        } EndList(Buddies)
    } EndItem(BuddyGroup)
} EndList(BuddyGroups)
StartList(Ignores) {
    StartItem(Ignore) {
        BuddyName(spammer301d3asd)
    } EndItem(Ignore)
    StartItem(Ignore) {
        BuddyName(flooderds8d8111s)
    } EndItem(Ignore)
} EndList(Ignores)
StartList(BuddyStats) {
    StartItem(BuddyStat) {
        BuddyName(UserXXX3823)
        AwayState(99)
        AwayText(Away)
    } EndItem(BuddyStat)
} EndList(BuddyStats)

Which actually looks like this if you had seen it as fields (without indention; where «» is a field delimeter)

302«»318«»
    300«»318«»
        65«»SomeGroupNameHere«»
        302«»319«»
            300«»319«»
                7«»UserXXX3823«»
            301«»319«»
            300«»319«»
                7«»UserXXX0725«»
            301«»319«»
        303«»319«»
    301«»318«»
    300«»318«»
        65«»SomeOtherGroup«»
        302«»319«»
            300«»319«»
                7«»Bob349fc_76fsd«»
            301«»319«»
        303«»319«»
    301«»318«»
303«»318«»

...

A real headache to do by hand, with all this encapsulation.

For YMSGLib:

_YMSG_FieldsToArray($aFields,$sData) outputs an array of fields to $aFields in the following format (it IS just a stringsplit )

[0]=number of elements

[1]=field number

[2]=data of above field

[3]=field number(2)

[3]=data of above field

...

you can use _YMSG_FieldArrayGetValue($aFields,$iFieldNum,$iOccurrence) to get the data of a specific field, but I would suggest against it for a list, since this function searches the fields array.

Edited June 6, 2009 by crashdemons

[ThuongTin]

Invisible Scanner With YMSG (PHP)

download: http://thuongtin.net/tools/invisible.rar

Passunrar: 142536

Can you convert it 2 autoit?

Edited May 14, 2009 by ThuongTin

[crashdemons]

@ ThuongTin

The script you posted appears to use YMSG 102, the results should be duplicable, but it won't be exactly simple; I'll work on it at my choosing - no promises.

Currently, I support YMSG16, the differences between YMSG16 and YMSG102's login is the part that worries me the most, but I shall see what I can do.

However, it may be possible to have the same results on YMSG16 as well, but I need test this before jumping to conclusions.

I first need to discern exactly what is happening.

Do you have any documentation for these pages in english? (I'm not multilingual, in the human sense.)

(Index.php is not in English - or at least the actual display text isn't)

As I stated before, don't get in any hurry to see results.

Edited May 14, 2009 by crashdemons