Jump to content

OpenCV v4 UDF


smbape
 Share

Recommended Posts

Hi again @Lion66

The theory was correct.
Using cv::convexHull, it is possible to determine if a polygone is concave.

Local $hull = _VectorOfPointCreate()
Local $o_arr_hull = _cveOutputArrayFromVectorOfPoint($hull)
_cveConvexHull($i_arr_dst_int32, $o_arr_hull)
For $i = 0 To _VectorOfPointGetSize($dst_int32) - 1
    _VectorOfPointGetItemPtr($dst_int32, $i, $tPointFPtr)
    Local $dst_int32_i = DllStructCreate($tagCvPoint, $tPointFPtr.value)

    _VectorOfPointGetItemPtr($hull, $i, $tPointFPtr)
    Local $hull_i = DllStructCreate($tagCvPoint, $tPointFPtr.value)

    If $hull_i.x <> $dst_int32_i.x Or $hull_i.y <> $dst_int32_i.y Then
        ConsoleWrite("Concave contour" & @CRLF)
        ExitLoop
    EndIf
Next
_cveOutputArrayRelease($o_arr_hull)
_VectorOfPointRelease($hull)


See the attached code.

160732-opencv-udf.au3

Edited by smbape
Link to comment
Share on other sites

I do not cease to be surprised at your fantastic knowledge! Of course it works!

It will take me a few days to look and try to understand, I am completely confused in the code options. I'll have questions later.

Thank you.

Link to comment
Share on other sites

I have added your UDF to the wiki :)

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2022-02-19 - Version 1.6.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (NEW 2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to comment
Share on other sites

Hello @ smbape

Congratulations! 👏 👏 👏 Now you are in a wiki (as I said!). It's worthy of the job you're doing!

I found this example. He draws the frame so beautifully.

I see that here the method of finding the contour is applied here and after that boundingRect.
I wanted to try, but I can't handle it myself.
Perhaps you will have the opportunity to show how to do this.

Attached is an example on the python.

Thank you so much.

 

Result.png

card.py

 

 

 

 

 

Edited by Lion66
Link to comment
Share on other sites

Hi @Lion66

Sorry for the late response, I was on another project.
I read the python code.

The detection is done in the following steps:

  • Detect objects of a scene using cv2.findContours.
    To improve the countour detection, the image is first blured with cv2.GaussianBlur, thus removing noises,
    then thresholded with cv2.threshold to enhance seperation of areas.
  • For each countour, pass it through a bank of images (images/cards/sample/)
  • The type of image that matches the most will be choosed as the type of the countour.

This technique will work if the background and the object are very contrasted.
If not, threshold will either blacked everything or whited everything, thus making countour detection impossible.

On 8/18/2021 at 9:07 PM, Lion66 said:

I wanted to try, but I can't handle it myself.

What do you  want to try?

  • Countour detection?
  • GaussianBlur?
  • Threshold?
  • knnMatch ?
  • A combination of those?

What have you tried that was not working?

Do you have a python/autoit example of your try?

Link to comment
Share on other sites

A very important part when using your UDF is to correctly declare the type of variables and structures before applying the function.
I have little understanding of the structure of the DLL, and I lack instructions for translating the syntax from docs.opencv.org to this UDF.
My plan is:

_cveGaussianBlurMat($gray_obj, $Gaus_obj, $ksize, 0, 0, $CV_BORDER_DEFAULT)
_cveThresholdMat($gray_obj, $gray_obj, $thresh, $maxval, $type)
_cveFindContoursMat($matImage, $matContours, $matHierarchy, $mode, $method, $offset = _cvPoint())
_cveBoundingRectangleMat($matPoints, $boundingRect)

And already the first function fails.

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
Opt("MustDeclareVars", 1)
#include "emgucv-autoit-bindings\cve_extra.au3"
_OpenCV_DLLOpen("libemgucv-windesktop-4.5.3.4721\libs\x64\cvextern.dll")

Local $img_obj = _cveImreadAndCheck("pic1\box.png", $CV_IMREAD_COLOR)   ; query image (small object)
Local $gray_obj = _cveMatCreate()
_cveCvtColorMat($img_obj, $gray_obj, $CV_COLOR_BGR2GRAY)

;_cveGaussianBlurMat($matSrc, $matDst, $ksize, $sigmaX, $sigmaY = 0, $borderType = $CV_BORDER_DEFAULT)
Local $Gaus_obj = _cveMatCreate()
Local $ksize = "(3,3)"
_cveGaussianBlurMat($gray_obj, $Gaus_obj, $ksize, 3, 0, $CV_BORDER_DEFAULT)

;_cveThresholdMat($gray_obj, $gray_obj, $thresh, $maxval, $type)
;_cveFindContoursMat($matImage, $matContours, $matHierarchy, $mode, $method, $offset = _cvPoint())
;_cveBoundingRectangleMat($matPoints, $boundingRect)

_cveImshowMat("", $Gaus_obj)
_cveWaitKey()
_cveDestroyAllWindows()
_Opencv_DLLClose()

And another question:
Can I process the error?
In the compiled script, if possible to show the message to the user instead of emergency completion?

Edited by Lion66
Link to comment
Share on other sites

Hi @Lion66

3 hours ago, Lion66 said:

A very important part when using your UDF is to correctly declare the type of variables and structures before applying the function.

You are right. I am aware of the high level of knwoledge needed in order to use the UDF correctly.
I planned on making a documentation for rules of translation from c++ to the UDF, however is it hard teach it correctly.

This is what I do to convert to the UDF:

  • Look at the udf definition without "Mat" at the end. For cv2.gaussianBlur it will be _cveGaussianBlur
  • The first line of the function is the c++ signature of the function, which is
    CVAPI(void) cveGaussianBlur(cv::_InputArray* src, cv::_OutputArray* dst, CvSize* ksize, double sigmaX, double sigmaY, int borderType);

     

  • The "Mat" version is a function where every cv::_InputArray, _OutputArray, _InputOutArray are derived from a matrix
  • CvSize* is a _cvSize() element. In this case Local $ksize = _cvSize(3,3)
  • When in the c++ exemple one of the cv::_*Array is a cv::Scalar, the "Mat" version cannot be used.
    You have to first create a cv::Scalar with _cveScalarCreate, then use _cve*ArrayFromScalar to convert it to the expected type before using it in the function
  • There are cases where the function has extra parameters required. Those parameters take their default value if not specified.
    If the UDF is not aware of those default values, you have to put them manually.
    In that case, you have to look at the documentation at opencv.
    For exemple _cveKAZEDetectorCreate(False, False, 0.001, 4, 4, $CV_KAZE_DIFF_PM_G2, $tFeature2DPtr, $tSharedPtr) comes from cv::KAZE::create

With the previous instructions, can you try again?

Not that I do not want to help, but I want to see if my instructions are understandable and correct.

Edited by smbape
Link to comment
Share on other sites

3 hours ago, Lion66 said:

Can I process the error?

When calling a dll, AutoIt doesn't provide a way for it. It just crashes.
The best things you can do are:

  • Tools > "Trace: Add Trace Lines"
  • Set $_cve_debug = 1
    This will print all successfully executed functions.
    With the last executed function, you can manage to find where it crashes.
    Crashes are usually due to incorrect types or Null pointers.
Edited by smbape
Link to comment
Share on other sites

Thank you for your answers.

The topic is very extensive. Sorry, if I to be importunate.

I advance with "$ksize = _cvSize(3,3)", but I did not understand why it was told about:

2 hours ago, smbape said:

You have to first create a cv::Scalar with _cveScalarCreate, then use _cve*ArrayFromScalar to convert it to the expected type before usin it in the function

Now I'm stuck on the "$matHierarchy" (third parameter in _cveFindContoursMat)

Local $img_scene = _cveImreadAndCheck("pic1\cards.JPG", $CV_IMREAD_COLOR)   ; query image - scene
Local $gray_scene = _cveMatCreate()
_cveCvtColorMat($img_scene, $gray_scene, $CV_COLOR_BGR2GRAY)

Local $Gaus_scene = _cveMatCreate()
_cveGaussianBlurMat($gray_scene, $Gaus_scene, _cvSize(3,3), 0, 0)
_cveThresholdMat($Gaus_scene, $Gaus_scene, 200, 255, $CV_THRESH_BINARY)


Local $H = _cveMatCreate()
Local $o_arr_H = _cveOutputArrayFromMat($H)
_cveFindContoursMat($Gaus_scene, $Gaus_scene, $o_arr_H, $CV_RETR_EXTERNAL, $CV_CHAIN_APPROX_SIMPLE)

 

Edited by Lion66
Link to comment
Share on other sites

1 hour ago, Lion66 said:

The topic is very extensive. Sorry, if I to be importunate.

I don't feel that way. It is a pleasure to have someone interested in my project.
That is why I am answering every question you have.

 

1 hour ago, Lion66 said:

but I did not understand why it was told about

It doesn't apply to this case.
You missed the first line, "When in the c++ exemple one of the cv::_*Array is a cv::Scalar, the "Mat" version cannot be used.".
For example, in Threshold_inRange.cpp#L89 , the method inRange takes a Scalar as a parameter. I was talking about those kind of cases.
The are also case where there are vectors like in SURF_FLANN_matching_homography_Demo.cpp#L73 . Again, _cve*ArrayFrom* should be used like in matching_homography_Demo.au3#L301

 

1 hour ago, Lion66 said:

Now I'm stuck on the "$matHierarchy":

It is more than that.
I you look at the documentation of cv::findContours, it is written

Quote
contours Detected contours. Each contour is stored as a vector of points (e.g. std::vector<std::vector<cv::Point> >).
hierarchy Optional output vector (e.g. std::vector<cv::Vec4i>), containing information about the image topology. It has as many elements as the

For contours, it is not a matrix, but a vector of points (std::vector<std::vector<cv::Point> >), that means, the "Mat" version cannot be used.
For hierarchy, it is an output vector (std::vector<cv::Vec4i>). Vec4i is a matrix. Therefore, hierarchy is a cv::_OutputArray from a vector of matrix.
It will then be

$contoursVect = _VectorOfVectorOfPointCreate()
$contours = _cveOutputArrayFromVectorOfVectorOfPoint($contourVect)

$hierarchyVect = _VectorOfMatCreate()
$hierarchy = _cveOutputArrayFromVectorOfMat($hierarchyVect)

; rest of the code here

; Then loop through the VectorOfVectorOfPoint like in
; https://github.com/smbape/node-emgucv-autoit-generator/blob/master/samples/tutorial_code/features2D/feature_homography/matching_homography_Demo.au3#L232

It is true that it needs c++ understanding.
If my directions are not enough to help you have a working script, come back to me

Edited by smbape
Link to comment
Share on other sites

I do not have knowledge in C++.

For academic purposes, your UDF is very good.

For practical purposes, I would transfer the creation of all auxiliary variables (necessary for the function itself) into UDF.

And the user would be given simple functions with user arguments.

But this is another story!

I have to admit I don't understand your explanation at this level. I'm sorry. :(

I myself would like to understand how to get a ready-made code.

But this is not my level and we are unlikely to advance like that.

It doesn't work again. :mad:

Local $contoursVect = _VectorOfVectorOfPointCreate()
Local $contours = _cveOutputArrayFromVectorOfVectorOfPoint($contoursVect)

Local $hierarchyVect = _VectorOfMatCreate()
Local $hierarchy = _cveOutputArrayFromVectorOfMat($hierarchyVect)

_cveFindContoursMat($Gaus_scene, $contours, $hierarchy, $CV_RETR_EXTERNAL, $CV_CHAIN_APPROX_SIMPLE)

 

Link to comment
Share on other sites

1 hour ago, Lion66 said:

For practical purposes, I would transfer the creation of all auxiliary variables (necessary for the function itself) into UDF.

And the user would be given simple functions with user arguments.

I agree with you.
However, OpenCV has more that 3000 functions.
It is impossible, without spending months, to translate everything.
And supposed I am done. Tracking updates from the point I started translation will be tedious.
Like every big c/c++ library out there, people just give up on using them with AutoIt, or write dlls, which requires c/c++ knowledge.

 

1 hour ago, Lion66 said:

But this is not my level and we are unlikely to advance like that.

It doesn't work again.

I understand your frustration. I will stop here.
Can you post your current file and image.
I will fix it for you.

Edited by smbape
Link to comment
Share on other sites

Like in cards.py, boundingRect has to be called on each countour.
The documentation of boundingRect only has one parameter and returns a cv::Rect.
Usually the UDF has no return value and the last parameter will receive the returned value.

Changes have been made in SearchContureDraw-v2.au3

Following cards.py, to crop an image based on a cv::Rect do:

$boundingRect.x -= 15
$boundingRect.y -= 15
$boundingRect.width += 15
$boundingRect.height += 15
$cropped = _cveMatCreateFromRect($img_scene, $boundingRect)

SearchContureDraw-v2.au3

Edited by smbape
Link to comment
Share on other sites

Hi @smbape

First thanks for your help and for your patience!
In the script, I want to recognize only one image, and changing the size of the frame is not important.
I don't know how to properly link the result after Filter matches and the frame.
I came up with an option, but for some reason the loop on line 71 makes only one turn.

 

SearchContureDraw-v5.au3

Link to comment
Share on other sites

You used the same variable in two different loops

; #71
For $i = 0 To _VectorOfVectorOfPointGetSize($contoursVect) - 1

; #134
For $i = 0 To _VectorOfVectorOfDMatchGetSize($knn_matches) - 1

; change it to
For $j = 0 To _VectorOfVectorOfDMatchGetSize($knn_matches) - 1
        _VectorOfVectorOfDMatchGetItemPtr($knn_matches, $j, $tVectorDMatchPtr)

 

Edited by smbape
Link to comment
Share on other sites

  • smbape changed the title to OpenCV v4 UDF

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...