Skip to content

mAP #1

Open
Open
mAP#1
@GOATmessi8

Description

@GOATmessi8

Have you ever evaluate the transformed trained model in VOC2007? I've tried your code and got a 71.9 mAP while the original is 76.8. Then I found a tiny error in test code, after fixing the result up to 72.8 mAP, still not enough...

Activity

longcw

longcw commented on Mar 4, 2017

@longcw
Owner

Yes, I got the same result. You can make a pull request for me to fix the bug.
I have no idea about the low mAP of my implementation. Did you try darknet implemented by the author?

GOATmessi8

GOATmessi8 commented on Mar 4, 2017

@GOATmessi8
Author

Not yet, but I found an issue in darkflow, it seems the transfer to tensorflow also cause some difference. https://github.com/thtrieu/darkflow/issues/25
I will make a pull request if I figure out the training part. Maybe that could solve the problem...

longcw

longcw commented on Mar 7, 2017

@longcw
Owner

I implemented the loss function following the darknet and the training process is work now.
I trained it on VOC2007 trainval set and got a 71.86 mAP ~50mAP on the test set.
Maybe you can find out some other problems about the low mAP with the help of darknet source code.

terrychenism

terrychenism commented on Mar 7, 2017

@terrychenism

@longcw Thank you for sharing code. I have tested the converted darknet model, which got ~72 mAP. Then I trained VOC07 trainval set for 160 epoch (totally use your github codes), which only got ~50 mAP. Did you successfully train the yolo2 detector?

longcw

longcw commented on Mar 8, 2017

@longcw
Owner

Thank you for your comment.
I tested the trained model and got the same result, ~50mAP. There are still some bugs for training. I am sorry for this.

crazylyf

crazylyf commented on Mar 14, 2017

@crazylyf
Contributor

For test phase, there are two parameters inconsistent with the original darknet:

  • The thresh parameter for bbox filtering is 0.001 in darknet, while it is 0.01 in test.py;
  • The iou_thresh for nms is 0.5 in darknet, while it is 0.3 in this project.
    For train phase, the thresh in cfgs/config.py should be 0.24, instead of 0.3;

As @ruinmessi , before correcting those parameters, the mAP in VOC2007-test is 71.9. Correction of first parameter improves slightly to 72.2, and correction the iou_thresh further boosts to 73.6.
The tensorflow version of yolo (darkflow) seems to suffer such a problem too, and an issue of that project pointed out some possible reasons. Maybe the reasons exist also in this project?

crazylyf

crazylyf commented on Mar 14, 2017

@crazylyf
Contributor

@ruinmessi What error in test code have you fixed?

GOATmessi8

GOATmessi8 commented on Mar 15, 2017

@GOATmessi8
Author

@longcw @crazylyf Sorry for leaving a long time. I boost the mAP to 74.3 by changing the nms order like this while this project do the nms in a function called postprocess. with the exact parameters you mentioned.

crazylyf

crazylyf commented on Mar 15, 2017

@crazylyf
Contributor

Why your mAP is 0.7 higher if we are using the same parameters? Am I missing something?

GOATmessi8

GOATmessi8 commented on Mar 15, 2017

@GOATmessi8
Author

The nms should implement before thresh holding.

longcw

longcw commented on Mar 15, 2017

@longcw
Owner

@ruinmessi Thank you for pointing out this problem.

GOATmessi8

GOATmessi8 commented on Mar 16, 2017

@GOATmessi8
Author

@longcw I am curious about how to convert the original weights to h5 file, could you please show me some details or scripts?

longcw

longcw commented on Mar 16, 2017

@longcw
Owner

@ruinmessi I use darkflow to load original weights from the binary weights file.

rdfong

rdfong commented on Apr 14, 2017

@rdfong

Is there any update on the training issue?

18 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @JesseYang@rdfong@xuzijian@terrychenism@GOATmessi8

        Issue actions

          mAP · Issue #1 · longcw/yolo2-pytorch