Jump to content

Recommended Posts

The goal here is to get the Internet History for endpoints and users in the enterprise to find what business use websites use Java.

When opening index.dat in Notepad, I can see all the data I need. Unfortunatly, trying to read that data in AutoIT is proving diffucult.

I've tried :

Func _ParseIndexdat($indexdatpath)
; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($Bindexdat)
$strIndexdat = BinaryToString($indexdat,2)
MsgBox(0,"String from Binary",$strIndexdat)
$FileArray = StringSplit($strIndexdat,@CRLF)
;This may get complex...
Dim $r = 1 ; to count the records
Dim $e = 0; to count the entries

; Start reading from line 1
For $line In $FileArray
$content = StringStripWS($line,7)
  Select
   Case StringInStr($line,"REDR")
    ; this is the start of a record
    MsgBox(0,"REDR",$content)
   Case StringInStr($line,"URL")
    ; this may be the start of a record
    MsgBox(0,"URL",$content)
   Case StringInStr($line,"LEAK")
    ; this is the start of an error record
    ; I will likely ignore it.
    MsgBox(0,"LEAK",$content)
   Case StringInStr($line,"http://")
    ; this is the line with a couple entries
    MsgBox(0,"http",$content)
   Case StringInStr($line,"Content-Type:")
    ; this is an entry I want
    MsgBox(0,"Content-Type",$content)
   Case StringInStr($line,"X-Powered-By:")
    ; this is an entry I want
    MsgBox(0,"Powered-By",$content)
   Case StringInStr($line,"~U:")
    ; this is an entry I want and it marks the end of a record
    MsgBox(0,"~U",$content)
   Case Else
    ; do nothing with the line
  EndSelect
Next
EndFunc

That doesn't get the info... That code doesn't return anything.

Using

$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($Bindexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$strIndexdat)

the "string from binary" msg box shows a LOT of data... but URL and http case are both blank or gibberish. BinaryToString($indexdat,2) and ,3 return Nothing in the cases - 4 does the same as 1.

Using :

$Bindexdat = FileOpen($indexdatpath, 16)
$indexdat = FileRead($Bindexdat);$indexdatpath)
MsgBox(0,"Index dat",$indexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat,@CRLF)

Shows that $indexdat and $strIndexdat are effectively the same.

Using :

; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
;$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($indexdatpath);$Bindexdat)
;$strIndexdat = BinaryToString($indexdat,4)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($indexdat,@CRLF)

It reads a lot more info... Content-Type is fine. ~U: is fine. URL is mostly blank.

Case http:// it will pop up the msg box, but $content is blank. So the data is there... I just can't figure out how to get it into a string.

Using :

$Bindexdat = FileOpen($indexdatpath,256)
$indexdat = FileRead($Bindexdat);$indexdatpath)
MsgBox(0,"Index dat",$indexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat,@CRLF)

Returns data, but URL and Http:// pop up as blank...

$Bindexdat = FileOpen($indexdatpath,48) - URL and Http are blank.

I've found a VBScript that is supposed to read the files... so far no luck on Win7 - it can't find index.dat file... lol! On remote XP machines, it can't find a history folder. So I don't know if it DOES read index.dat files.

I've attached it.

Any ideas or code help is greatly appreciated!!

IE_Network - Copy.txt

Edited by Graywalker
Link to post
Share on other sites

I altered the vbscript to point directly to the index.dat file and it gets info, but returns all kinds of gibberish.

... that totally messes up the reply even when pasted as code!

So, I've attached it as a screen cap.

post-38206-0-27179900-1327335952_thumb.p

Edited by Graywalker
Link to post
Share on other sites

Okay, I've found out that the data IS there. I've just got to find a way to get TO it.

Func _ParseIndexdat($indexdatpath)
; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
$Bindexdat = FileOpen($indexdatpath, 16)
$indexdat = FileRead($Bindexdat);$indexdatpath)
;MsgBox(0, "Index dat", $indexdat)
$strIndexdat = BinaryToString($indexdat, 1)
$strIndexdat = StringStripWS($strIndexdat, 7)
;MsgBox(0, "String from Binary", $strIndexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat, @CRLF)
;This may get complex...
Dim $r = 1 ; to count the records
Dim $e = 0; to count the entries
; Start reading from line 1
For $line In $FileArray
  $line = StringReplace($line, @CRLF, "")
  $line = StringReplace($line, @CR, "")
  $line = StringReplace($line, @LF, "")
  Select
   Case StringInStr($line, "http://")
    ; this is the line with a couple entries
    ;MsgBox(0, "http", $line)
    ; Check to see if URL is in the line
    If StringInStr($line, "URL") Then
     ; it is a REDR or LEAK, trim to URL
     $urlpos = StringInStr($line, "URL")
     $line = StringTrimLeft($line, $urlpos)
    EndIf
    $httppos = StringInStr($line, "http")
    $line = StringTrimLeft($line, $httppos - 1)
    $dotpos = StringInStr($line, ".", "", 3)
    If $dotpos > 10 Then
     $content = $line
     $linelen = StringLen($line)
     $trimfromright = $linelen - ($dotpos + 3)
     $line = StringTrimRight($line, $trimfromright)
     ;MsgBox(0,"Trim", "string lenght : " & $linelen & @CRLF & "dot position: " & $dotpos & @CRLF & "Trim from Right : " & _
     ;$trimfromright & @CRLF & $line)
    Else
     ; nothing
    EndIf
    $record = $line & ","
   Case StringInStr($line, "Content-Type:")
    ; this is an entry I want
    $line = StringStripWS($line, 7)
    $record = $record & $line & ","
   Case StringInStr($line, "X-Powered-By:")
    ; this is an entry I want
    $line = StringStripWS($line, 7)
    $record = $record & $line & ","
   Case StringInStr($line, "~U:")
    ; this is an entry I want and it marks the end of a record
    $line = StringReplace($line, "~U:", "")
    $line = StringStripWS($line, 7)
    $record = $record & $line
    FileWriteLine($logfile, $record)
    $record = ""
   Case Else
    ; do nothing with the line
  EndSelect
Next
EndFunc   ;==>_ParseIndexdat

I am getting fairly spotty results - leaving in lots of random characters before and after the URLs.

Could anyone come up with a way to use StringRegExp to pull the URLs - all of them - from a line?

I've tried several from : http://regexlib.com/Search.aspx?k=URL&AspxAutoDetectCookieSupport=1

with no consistent luck.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Colduction
      Hi AutoIt Programmers, i wanna figure out how to use Binary functions in C# like:

      BinaryMid
      BinaryLen
      IsBinary and other basic ones were too ez, but those two were hard to noob like me.

      I appreciate for your helps/hints.
    • By TheSaint
      An adaption of an adaption.
      A good while back, I created a program, KindEbook Wishlist, that I use most days and is still available here at the forum. It works well, keeping tabs on price changes for Kindle ebooks at Amazon ... at least for my modest wants.
      About a year or so later, I struck upon the idea of adapting that program for CDs, DVDs, Blu-rays etc from the JB Hifi store, where there was a limit of about 50 items on the in-store wishlist. JB Wishlist is kind of a niche program, really only suitable for those from AUS, at the AutoIt Forum, who would want such a thing ... so not many I imagine, so it has never been available here. However, if you meet that criteria etc, and are willing to register at the AutoIt4Life Clubrooms, then it is available in the Project Chat section there ... though not the more recent updates (you need to ask me about them). That too, works quite well for my modest needs, and even includes an additional Preview & Summary window, which KindEbook Wishlist doesn't have ... though both programs do have access to a Details window per item, that is like the Preview window ... just doesn't constantly display alongside the Main program window (see later for an example with IonGoG Wishlist).
      So, now we come to IonGoG Wishlist, which is an adaption of the JB Wishlist program, and used for keeping tabs on GOG Games ... prices, history of changes, bought list, etc.
      IonGoG Wishlist is an incomplete adaption, so still in beta ... though all the really important elements work well enough. I have been working on it in dribs and drabs, and I now think it has come to a reasonable enough stage to share it ... just be advised, that some of the features accessed via the right-click menu etc, may not work or give strange results. Any of the options that I have given an Accelerator key facility to, should be working fine ... and most of the others are too ... and I am not even sure at this point, of what remains to be adapted ... not much I imagine.
      The name by the way, in case you were wondering, is a phonetic mangling on the words - keep an Eye On your GOG Wishlist.
      Here is a screenshot, with the Preview window on left. The Preview window can be placed at right instead or even turned off.

      Download, includes source files.
      IonGoG Wishlist v0.0_b28.zip  (see detail at Post#23)
      Enjoy!
      OLDER DOWNLOADS
      SUPPLEMENTARY
      I have another program that GOG users may be interested in, that can assist with getting game images that IonGoG Wishlist gets and doesn't get ... larger versions taken from modified thumbnail links in their GOG Library.
      Downloads Dropbox  (see the GOG example in Post #7)
      ADDENDUM
      In reality, compared to most stores I have come across, the GOG store is great, well setup and laid out. and for most things, including their Wishlist, is more than adequate. So I rarely do a full list Price Query ... so much quicker and easier to just check the two pages of my in-store wishlist. When I first started work on IonGoG Wishlist, the in-store wishlist wasn't as good as it is now.
      All that said, I still find it handy to use IonGoG Wishlist, for a variety of things - Offline browsing, Cover images to use with my bought & downloaded game folders, price changes history (and patterns of GOG behavior for sales etc), a bought list (with price I paid and date etc), Game Notes & Warnings, etc. And while not as quick as just checking the pages of your in-store wishlist, you can just set the full, favorites or non-favorites Query going, while you go away and do something else, and then later come back and look at the highlighted changes that may have occurred.
    • By Mariog2000
      I've tried for a few days to simply open a url in IE and click an IE object that I can't "TAB" to and don't want to use a mouse-click if at all possible.  I've read up on things like using IEAction and IETagNameGetCollection to find the object but not smart enough to figure out how that works with my url or how to tell if my url is "basic" or a "form", "frameset", "iframe", etc.
      The other problem is I've tried using the "Run" and "ShellExecute" commands to open the browser, which works, but it opens it in a way that doesn't include my currently logged in user account.  For example, if I just click IE or Chrome for that matter and open my default homepage of google, the page that opens knows I'm logged in.  When I use "Send" "ShellExecute" or "_IE_Create" they always open pages generically which would require a login.
      So questions are, if I have a URL I'm trying to open, that's passes the currently logged on users credentials when opening the page (no clue where that happens)
      1.  Are there any code examples someone is willing to share that opens my IE maximized, with logged in credentials, and
      2.  Once that works, can someone suggest a technique a non-developer can find which IE Objects I need to send a "click" to so it opens the correct menu, pop-up, url, etc., associated with that object.  I've copied what doesn't work and sure I'm way off but it's not for a lack of effort. I'll keep trying and post updates but not looking good 🙂
      #include <IE.au3>
      Call ("selection")
      Func selection ()
      Global $oIE = _IECreate ("https://internal-webpage.aspx")
      Local $selectid = _IEGetObjByName ($oIE, "signageNo")
      Local $button = _IEGetObjByName ($oIE, "GO")

      _IEFormElementOptionSelect ($selectid, "3")
      _IEAction ($button,"click")
      EndFunc
      #include <IE.au3> Call ("selection") Func selection () Global $oIE = _IECreate ("https://internal-webpage.aspx") Local $selectid = _IEGetObjByName ($oIE, "signageNo") Local $button = _IEGetObjByName ($oIE, "GO") _IEFormElementOptionSelect ($selectid, "3") _IEAction ($button,"click") EndFunc 1. 
    • By Yodavish
      My AutoIt script works fine, for the most part, however, if it goes idle for roughly 30 minutes, it will won't be able to send data to a 3rd party application window titled, "Case #". 
      The problem lies within the function "SendFusion", it's able to see the handle for the "WinActivate" and "WinWaitActivate" (I've also tried putting in 10 seconds in the WinWaitActivate as well, doesn't work). However, if we have been idle, it will always fail to send data afterward. On what I've tried so far:
      "Send" function with the data and a {ENTER} i.e. Send($CaseTxt & "{ENTER}"), this won't even hit the enter key  ControlSetText, tried passing the handle from WinActivate, using the title "Case #" that was copied directly from the Autoit Window Info, hard coding the title, Using CLASS:WindowsForms10... always returns a 0 ControlCommand (same as above) always returns a 0 ControlGetFocus always returns a 0 ControlFocus, always returns a 0 Creating a new and simple Au3 script that just sends data to "Case #", also fails NOTE: If I close the Au3 script and just scan the data directly into the "Case #" or even type it, it works fine with no issues. I'm confused as to why it would not work after a long idle period?
      Below are the essential parts of the script, I can provide the entire thing if needed. Any suggestions would be greatly appreciated!
      #include <AutoItConstants.au3> #include <GUIConstantsEx.au3> #include <MsgBoxConstants.au3> #include <Process.au3> #include <Misc.au3> #include <IE.au3> ;--------------------------------------------------------------- ; Only one instance can run ;--------------------------------------------------------------- If _Singleton("gross.exe", 1) = 0 Then MsgBox($MB_SYSTEMMODAL, "Warning", "Gross.exe is already running. Please exit the existing version first (check the icons in the lower right corner of your screen) before running it again.") Exit EndIf ;--------------------------------------------------------------- ; AutoIt Options ;--------------------------------------------------------------- Opt("GUIOnEventMode", 1) ; Change to OnEvent mode Opt("WinWaitDelay", 0) ; Alters how long a script should briefly pause after a successful window-related operation. Time in milliseconds to pause (default=250). Opt("WinTextMatchMode", 1) ; Alters the method that is used to match window text during search operations. 2 = Quick mode Opt("SendKeyDelay", 0) ; Alters the length of the brief pause in between sent keystrokes. A value of 0 removes the delay completely. Time in milliseconds to pause (default=5). if ($fusionWindowTitle="NONE" or $fusionCaseNumWinTitle="NONE" or $appWindowTitle="NONE" or $winActivateTimeout="NONE" or $winCloseTimeout="NONE" or $winWaitTimeout="NONE" or $pdfViewerWidth="NONE" or $pdfViewerHeight="NONE" or $pdfViewerLeft="NONE" or $pdfViewerWindowName="NONE" or $pdfViewerExeName="NONE" or $pdfUrl="NONE" or $pdfDownloadCommand="NONE") Then Msgbox(0,"ERROR", "gross.ini is missing or does not contain all settings. Please contact Help Desk with this information") Exit -1 EndIf ;--------------------------------------------------------------- ; application settings/constants ;--------------------------------------------------------------- Global Const $[VARIABLE] = IniRead("gross.ini", "default", "[TitleOfVariable]", "NONE") Global Const $[VARIABLE] = IniRead("gross.ini", "default", "[TitleOfVariable]", "NONE") Global Const $[VARIABLE] = IniRead("gross.ini", "default", "[TitleOfVariable]", "NONE") Global Const $[VARIABLE] = IniRead("gross.ini", "default", "[TitleOfVariable]", "NONE") ;--------------------------------------------------------------- ; create the main window ;--------------------------------------------------------------- Local $mainWindow = GUICreate($appWindowTitle, 380, 190) ; create the main GUI window GUISetOnEvent($GUI_EVENT_CLOSE, "handleCloseClick") ; when the click the windows close button call handleCloseClick() WinSetOnTop($appWindowTitle, '', 1) ; Sets main GUI always on top ;--------------------------------------------------------------- ; create the controls on the main window ;--------------------------------------------------------------- Local $okButton = GUICtrlCreateButton("OK", 160, 160, 54,24) ; create OK button GUICtrlSetOnEvent($okButton, "validateInput") ; when they click ok, call handleInputProcessRequest() GUICtrlCreateLabel("Scan Input", 10, 10) ; create a label Global $inputScan = GUICtrlCreateInput("", 10,134,358) ; create the input scan box GUICtrlSetOnEvent($inputScan, "validateInput") ; when they press enter in the scan input box, call handleInputProcessRequest() GUICtrlSetState($inputScan,$GUI_FOCUS) ; automatically sets focus on the input field GUISetState(@SW_SHOW) ; main GUI loop that runs at all times ;--------------------------------------------------------------- While 1 ;---------------------------------------------------------- ; If sumatraPDF active, resets focus back to GUI ;---------------------------------------------------------- If WinActive($pdfViewerWindowName) Then consoleWrite('While Loop sumatraPDF was active' & @CRLF) setMainWindow() ;---------------------------------------------------------- ; After data sent to ProTracker, check for mismatch ; save button to trigger the event handler when clicked ;---------------------------------------------------------- ElseIf WinActive($proTrackerWindowTitle) Then $ie = _IEAttach("ProTracker") $mismatchButton = _IEGetObjByName($ie, $proTrackerMisMatchButtonId) $oEvent = ObjEvent($mismatchButton, "mismatchButton_") If @error Then setMainWindow() ;--------------------------------------------------------------------- ; If Fusion is prompting for the case #, focus back to main GUI input ; If second GUI 'Unknown' detect do nothing ;--------------------------------------------------------------------- ElseIf WinExists($fusionWindowTitle) and WinExists($fusionCaseNumWinTitle) and Not WinActive($mainWindow) and Not WinExists('Unknown Input') Then setMainWindow() EndIF ;--------------------------------------------------------------------- ; Checks if $processFlag = Done, if so, shows PDF viewer and clears ; processFlag for next iteration ;--------------------------------------------------------------------- IF WinExists($pdfViewerWindowName) and $processFlag == 'Done' Then WinSetState ($pdfViewerWindowName, '', @SW_SHOW) $processFlag = '' setMainWindow() EndIF Sleep(100) ; Sleep to reduce CPU usage WEnd Func handleInputProcessRequest($input) $processFlag = True Local $hTimer = TimerInit() consoleWrite('> handleInputProcessRequest: ' & @CRLF) Local $idInput = identifyInput($input) ; Select a proccess to run based on id input Select Case $idInput = "container" ; regex that captures only the case number $caseTxt = StringUpper(StringRegExpReplace($strippedInputData, '\??(\w*\d*-\d*|\d*).*$','$1')) If WinExists($fusionWindowTitle) and WinExists($fusionCaseNumWinTitle) Then sendProTracker(StringUpper($strippedInputData)) sendFusion($caseTxt) ; gets pdf for sumatraPDF in seperate script, since that function is the slowest Run("viewPDF.exe " & $caseTxt) ;getSumatraPDF($caseTxt) $lastCaseNum = $caseTxt Else ; Checks to make sure the previous "Container" case num is the same ; to the current Container case num, before sending to ProTracker If($lastCaseNum <> $caseTxt) Then ; If not the same case number send alert sound SoundPlay("error.wav") Else sendProTracker($strippedInputData) EndIf EndIf Case $idInput = "user" sendProTracker($strippedInputData) Case $idInput = "cassette" sendProTracker(StringUpper($strippedInputData)) Case $idInput = "unknown" selectUnknown() EndSelect Local $fDiff = TimerDiff($hTimer) $processFlag = 'Done' consoleWrite('> handleInputProcessRequest Completed Total time: ' & $fDiff & ' ' &@CRLF&@CRLF&@CRLF) EndFunc Func sendFusion($caseTxt)     Local $hTimer = TimerInit()     consoleWrite('+ sendFusion initiated: ' & $caseTxt & @CRLF)     If $caseTxt = "" Then         MsgBox(0, "Error", "Not a valid case number")     Else         ;$fusionCaseNumWinTitle = "Case #"         $retVal1 = WinActivate($fusionCaseNumWinTitle, "")         consoleWrite("ReturnValue WinActivate " & $retVal1 & @CRLF)         $retVal2 = WinWaitActive($fusionCaseNumWinTitle,"",$winActivateTimeout)         consoleWrite("ReturnValue WinWaitActive: " & $retVal2 & @CRLF)         $retVal3 = ControlSetText($fusionCaseNumWinTitle, "", "", $caseTxt)         ;$retVal3 = ControlCommand($fusionCaseNumWinTitle, "", "", "EditPaste", $caseTxt)         consoleWrite("ReturnValue ControlSetText: " & $retVal3 & @CRLF)         ;Send($caseTxt & "{ENTER}")         WinWaitClose($fusionCaseNumWinTitle, "", $winCloseTimeout)         consoleWrite('+ sendFusion WinWaitClose: ' & @CRLF)     EndIf     Local $fDiff = TimerDiff($hTimer)     ConsoleWrite('+ sendFusion Completed Total time:' & $fDiff & ' ' & @CRLF) EndFunc
      Console logs from the SciTLE
      Window (x86) Info matches the correct handle for the WinActivate and WinWaitActivate

      Window Control (x86) Info, I've tried the CLASS, the ID, also I just noticed that the "handle" in "Control" and "Window" appear to be different as well.

       
      So far the only work-around is to close down the 3rd party application and the AutoIt script, re-open them and it works all completely fine. But this is a pain for the end-user since it's all touch screen and it slows down their workflow, which they can be extremely sensitive about.
       
    • By Stew
      (Edited from original.  Please note that I AM NOT AN AUTOIT EXPERT.  I write code using Autoit frequently but I am no expert, especially when it comes to I/O.  So any remarks that start with "Why did you..." can be answered by referring to the first sentence.  This project was done in Autoit because of an interface I built to display the data.)
      Attached is a program and ascii input file I wrote to read stock price data, convert it to binary and then read it back into the program in binary.  The goal was to show increased performance for reading the files in binary and provide a demo on how to read/write binary for int32, int64, double and strings for anyone who might find it helpful.  The results on my PC show the following:
      Time to read ascii file only: 456.981951167202
      Ascii read & process time: 6061.83075631701
      Binary write file time: 14787.9184635239
      Time just to read binary file: 42.418867292311
      Binary read and process time: 4515.16129830537
      A couple things to note:
      1) The 32 MB ascii file took 10x longer to read than the 15 MB binary file.  Not entirely sure why.  Both were read into a buffer.
      2) The Binary write takes a long time but I made no effort to optimize this because the plan was to write this file one time only so I don't mind if it takes longer to write this file.  I care much more about how long it takes to read the file because I will be reading it many times.
      3) There was a modest gain in converting the ascii file to binary in terms of file size and reading speed.
      So big picture... not sure it's worth the effort to convert the files to binary even though most of the data is numerical data in the binary file.  That was actually surprising as I expected there would be more of a difference.  Any ideas on how to get the binary data to read at a faster rate would be great.
       
      binary.au3
      2019_02_08.zip
×
×
  • Create New...