Sign in to follow this  
Followers 0
Chimp

How to view only rows with duplicate values in a table

4 posts in this topic

#1 ·  Posted (edited)

Referring to a question posted here, this is a possible SQL query that should do:

supposed you have an sqlite table named 'multimedia' with a column named 'sha1_hash', and you want to see only rows with the 'sha1_hash' value that appears more than one time within the table, you could use an SQL query like this:

SELECT sha1_hash, COUNT(sha1_hash) AS sha1_sum FROM multimedia GROUP BY sha1_hash HAVING sha1_sum>1 ORDER BY sha1_sum DESC;

this query should show only records with the value of sha1_hash that appears more times also in other records, showing it only one time, with a column indicating how many occurrences are present, ordered from the most recurring to the least recurring one.

I don't know if this query can be simplified a bit...?,

.... if there is a simpler way I would like some hint,
Thanks

Edited by Chimp
corrections in title and text

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

29 minutes ago, Danp2 said:

That's how I would do it. Not sure how you could get any simpler...

.... thanks .... :)

Edited by Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

What will happen to all items with same hash? Obviously there must remain one item and the rest deleted?

-- original

SELECT sha1_hash, COUNT(sha1_hash) AS sha1_sum FROM multimedia GROUP BY sha1_hash HAVING sha1_sum>1 ORDER BY sha1_sum DESC;

-- my take
--delete from multimedia
select sha1_hash from multimedia
where id not in ( select min(id)
from multimedia
group by sha1_hash )

Yes / No / Maybe ?

Edited by Skysnake

Skysnake

Why is the snake in the sky?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • argumentum
      By argumentum
      ..let's say I store in a mysql column "[1,2,11,12,123]".
      How can I make s query to get all those rows that is 1, or is 12, etc ?
      I'm using  mysqlnd 5.0.11-dev - 20120503 
      Any advise on what technique I should use is welcomed
      Thanks
      PS: ..I'm adding a comma to each end, as in  [,1,2,11,12,123,] and will search with like %,12,% .
      Not pretty but I'm in a hurry and can't come up with a better solution =/
    • sc4ry
      By sc4ry
      Dear all,
      I already have a formula which will select doublicates, but sometimes there might be entries with additional information starting after "-", which I now want to trim away.
      Current formula is:
      Select DISTINCT a.title, a.oid, a.size, a.disk from DB a, DB b where (a.title like b.title and a.oid <> b.oid)" & $ActiveDisks_Filter & " AND (a.comment <> 'Delete' AND b.comment <> 'Delete') Order by a.title ASC, a.size DESC; testwise I want to add a simple solution like left, but afterwards I get an error but I do not know why.
      Select DISTINCT a.title, a.oid, a.size, a.disk from DB a, DB b where (left(a.title,15) like b.title and a.oid <> b.oid) Order by a.title ASC, a.size DESC; Error: near "(": syntax error: Finally I would like to insert intstr to search for "-" but currently I am not able to extend my current code =/
      Thanks a lot for your help.
      EDIT: hey, I managed it, left was not working also it was somehow shown that the tool knowed the comment. I did it now with substr+instr
      (substr(a.title, 1, instr(a.title, ' - '))  
    • AndyS19
      By AndyS19
      I have code that does a WMI SQL query to find all defined printers, and I want to parse the returned object in several places.  However, after parsing it the first time, all other times fail to find any printer objects.
      Here is my test code:
      test() Func test() Local $oPrinters, $oPrinter, $err, $cnt, $oP, $query $query = "SELECT * FROM Win32_Printer" $oPrinters = doQuery($query) $err = @error LogMsg("+++: $err = " & $err & ", isObj($oPrinters) = " & IsObj($oPrinters)) If ($err == 0) Then LogMsg("FIRST LOOP") ; <=== FIRST LOOP $cnt = 0 $oP = $oPrinters LogMsg("+++: isObj($oP) = " & IsObj($oP)) For $oPrinter In $oP $cnt += 1 LogMsg("+++: isObj($oPrinter): " & IsObj($oPrinter) & ", $oPrinter.Name ==>" & $oPrinter.Name & "<==") Next LogMsg("+++: Found " & $cnt & " printers") LogMsg("SECOND LOOP") ; <== SECOND LOOP $cnt = 0 $oP = $oPrinters LogMsg("+++: isObj($oP) = " & IsObj($oP)) For $oPrinter In $oP $cnt += 1 LogMsg("+++: isObj($oPrinter): " & IsObj($oPrinter) & ", $oPrinter.Name ==>" & $oPrinter.Name & "<==") Next LogMsg("+++: Found " & $cnt & " printers") EndIf EndFunc ;==>test Func doQuery($sQuery, $lnum = @ScriptLineNumber) #forceref $lnum LogMsg("+++:" & $lnum & ": doQuery(" & '"' & $sQuery & '"' & ") entered") Local $oWMIService, $oResults, $errstr Local $wbemFlags = BitOR(0x20, 0x10) ; $wbemFlagReturnImmediately and wbemFlagForwardOnly $oWMIService = ObjGet("winmgmts:\\" & "localhost" & "\root\CIMV2") If (IsObj($oWMIService)) Then $oResults = $oWMIService.ExecQuery($sQuery, "WQL", $wbemFlags) If (IsObj($oResults)) Then LogMsg("+++: doQuery() returns @error = 0, Good: returning the object") Return (SetError(0, 0, $oResults)) ;;; Good: return the object Else $errstr = "" _ & "WMI Query failed." & @CRLF _ & "This is the query:" & @CRLF _ & " " & $sQuery LogMsg("+++: ====>" & $errstr & "<===") LogMsg("+++: doQuery() returns @error = 1") Return (SetError(1, 0, $errstr)) ; Error: Query faled EndIf Else $errstr = "" _ & "WMI Output" & @CRLF _ & "No WMI Objects Found for class: " & @CRLF _ & "Win32_PrinterDriver" & @CRLF _ & "using this query:" & @CRLF _ & " " & $sQuery LogMsg("+++: ====>" & $errstr & "<===") MsgBox(0, "ERROR", $errstr) ; Error: Cannot get $oWMIService object Exit (1) EndIf EndFunc ;==>doQuery Func LogMsg($msg, $lnum = @ScriptLineNumber) ConsoleWrite("+++:" & $lnum & ": " & $msg & @CRLF) EndFunc ;==>LogMsg Parsing the returned $oPrinters object shows 5 printers:
      +++:15: FIRST LOOP +++:18: +++: isObj($oP) = 1 +++:22: +++: isObj($oPrinter): 1, $oPrinter.Name ==>Microsoft XPS Document Writer<== +++:22: +++: isObj($oPrinter): 1, $oPrinter.Name ==>Microsoft Office Document Image Writer<== +++:22: +++: isObj($oPrinter): 1, $oPrinter.Name ==>Fax<== +++:22: +++: isObj($oPrinter): 1, $oPrinter.Name ==>Canon MG7100 series Printer WS<== +++:22: +++: isObj($oPrinter): 1, $oPrinter.Name ==>Canon MG6100 series Printer WS<== +++:24: +++: Found 5 printers Parsing it again, shows no printers:
      +++:26: SECOND LOOP +++:29: +++: isObj($oP) = 1 +++:35: +++: Found 0 printers  
    • GhostLine
      By GhostLine
      Hello guys !
      I'm stuck in something stupid (I guess), but since I've no clue on how to solve it, here I come
      I'm trying to collect the result of the query "select @@hostname" on MSSQL Server, but it doesn't work ... and I'm sure I'm connected to the database (I've successfully seen the database treeview). 

      #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=n #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include<_sql.au3> #include<array.au3> Local $aData,$iRows,$iColumns ;Server ID and credentials for tCards Global $ServerAddressT = "10.200.88.1" Global $ServerUserNameT = "user" Global $ServerPasswordT = "password" Global $DatabaseNameT = "DataBASE" ;Connect to DB _SQL_RegisterErrorHandler();register the error handler to prevent hard crash on COM error $OADODB = _SQL_Startup() If $OADODB = $SQL_ERROR Then MsgBox(0 + 16 + 262144, "Error", _SQL_GetErrMsg()) If _sql_Connect(-1, $ServerAddressT, $DatabaseNameT, $ServerUserNameT, $ServerPasswordT) = $SQL_ERROR Then     MsgBox(0 + 16 + 262144, "Error 1", _SQL_GetErrMsg())     _SQL_Close()     Exit Else     If _SQL_Execute(-1,"select @@SERVERNAME") = $SQL_ERROR then         Msgbox(0 + 16 +262144,"Error",_SQL_GetErrMsg())     Else         _ArrayDisplay(_SQL_GetTable(-1,"select @@SERVERNAME",$aData,$iRows,$iColumns)) ;~         $toto = _SQL_Execute(-1,"select @@SERVERNAME") ;~         _ArrayDisplay($toto)     EndIf EndIf
    • Gorby7
      By Gorby7
      I have an Autoit-based client GUI that uses a single MS SQL Server database. I have no problem connecting to and executing queries against this database from my AutoIT code, with the below exception:
      One of the database's stored procedures I need to execute requires a parameter that is a table. I've set up the appropriate table_type on the database and everything works fine on the database itself when I execute the stored proc with a table variable for the parameter (no AutoIT involved). The problem is that I can't figure out how to pass a table parameter from AutoIT, assuming it's even possible. The code below shows the two methods I've tried, both methods return a RecordSet object that is at EOF.
      #include <Array.au3> Opt("MustDeclareVars", 0) $sDBSrv = "ITSQL01.domain.com" $sDBName = "INVDEV" $sADOName = "ADODB.Connection" $oSQLConn = ObjCreate($sADOName) $sConnStr = "Driver={SQL Server};Server=" & $sDBSrv & ";Database=" & $sDBName & ";Trusted_Connection=yes;" $oSQLConn.Open ($sConnStr) $rsoFacFriendly = $oSQLConn.Execute("SELECT TOP 5 FriendlyName FROM Facility") If Not $rsoFacFriendly.EOF Then $aFacFriendly = $rsoFacFriendly.GetRows() _ArrayDisplay($aFacFriendly) $rsoSPResults = $oSQLConn.Execute("EXEC spTableParameterPassTest " & $aFacFriendly) Select Case $rsoSPResults.EOF = False MsgBox(0, "sp exec1", "not end of file") $aSPResults = $rsoSPResults.GetRows() _ArrayDisplay($aSPResults) Case IsObj($rsoSPResults) = 0 MsgBox(0, "sp exec1", "rso isn't even an object") Case $rsoSPResults.EOF = True MsgBox(0, "sp exec1", "At end of file") Case Else MsgBox(0, "sp exec1", "Something else happened") EndSelect $rsoSPResults = 0 $rsoSPResults = $oSQLConn.Execute("EXEC spTableParameterPassTest " & $rsoFacFriendly) Select Case $rsoSPResults.EOF = False MsgBox(0, "sp exec2", "not end of file") $aSPResults = $rsoSPResults.GetRows() _ArrayDisplay($aSPResults) Case IsObj($rsoSPResults) = 0 MsgBox(0, "sp exec2", "rso isn't even an object") Case $rsoSPResults.EOF = True MsgBox(0, "sp exec2", "At end of file") Case Else MsgBox(0, "sp exec2", "Something else happened") EndSelect $rsoSPResults = 0 $rsoFacFriendly = 0 $oSQLConn.Close $oSQLConn = 0 Exit