NCVPRIPG 2019 IDD Benchmark

Semantic segmentation on IDD Lite Dataset

This challenge is open to students in Indian universities and colleges. The challenge will use IDD-Lite, a dataset that is small and compact to fit on any personal computer and so will not huge-compute infrastructure. IDD-Lite is less than 50MB is size, contains 7 classes (compared to 30 in IDD).

Directions for Participation

  1. Register here for receiving updates about the competition.
  2. Register an account at the IDD website: http://idd.insaan.iiit.ac.in/
  3. After logging in to IDD website, download the IDD Lite dataset (http://idd.insaan.iiit.ac.in/dataset/download/)
  4. Segmentations masks have all been already generated for IDD Lite in the download.
  5. Once you have built a model, and have the predictions of the model in any of the split (train, val), you can evaluate the metric as directed here: https://github.com/AutoNUE/public-code#evaluation. Use the following command for segmentation evaluation:
    python evaluate/idd_lite_evaluate_mIoU.py --gts $GT --preds $PRED --num-workers $C
    Your predictions is a png image, which has the size of 256x128. Each pixel of this image contains the label as level1Ids (see labels code) of the corresponding image (resized to 256x128). The evaluation code above resizes both your prediction and ground truth png files to 256x128, in case they are not of that size.
  6. Finally you can upload the predictions for the test split, to be evaluated for the leaderboard here: http://idd.insaan.iiit.ac.in/evaluation/submission/submit/
Results
Team/Uploader Name Method Name mIoUmIoU for L1 IDs at 128p
kt002 trial 0.0321
SSN_CSE Modified FCN 0.3438
KLE Tech_Team CustomNet 0.4563
KLE Tech_Team CustomNet 0.0768
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.5017
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.2271
bekar_phd_students Unemployment ki shakti 0.4809
Deepsegmentation Deep Encoder Decoder Network for semantic segmetnation 0.5017
TCE RS Lab Sematic Segmentation U-net 0.5323
kz001 CNet 0.5232
TCE RS Lab Sematic Segmentation U-net 0.5432
SGGS-IITJ testing 0.5296
KLE Tech_Srikar CustomNet 0.0321
SGGS-IITJ Nahi batayenge 0.5586
SSN_CSE FCN 2 0.3771
KLE Tech_Team CustomNet 0.0781
kz002 CNet 0.5196
kz003 CNet 0.4802
test test 0.4748
t1 t2 0.4831
bekar_phd_students Unemployment ki shakti 0.5293
SSS Gated Shape CNN 0.4872
BharatAI BharatAI-t1 0.5262
SSN_CSE FCN 3 0.3137
Sanchayan Santra nnet 0.3235
Residual U-Net Residual U-Net2 0.5935
test test 0.3235
bekar_phd_students Unemployment ki shakti 0.4968
SSN_CSE UNet 0.1361
SSN_CSE FCN 0.0926
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.4738
bekar_phd_student bekar 0.4616
tyrion first_try 0.4043
SGGS-IITJ testing 0.5176
KLE Tech_Team CustomNet 0.538
SGGS-IITJ nahi-batayenge(testing) 0.5588
TCE RS Lab Sematic Segmentation U-net 0.532
First_Resnet Residual U-Net 0.5838
kt001 trial 0.0006
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.3986
Swetha B Enet 0.3986
mohit_asudani resnet 0.3885
Testing Testing 0.0959
testing testing 0.0959
tensorcat dlv3p 0.0031
mohit_asudani resnet 0.3885
USSB Try1 0.5614
SSN_CSE FCN 4 0.3407
kz004 CNet 0.4349
Parimal TBd 0.0611
Parimal TBd 0.0611
PK tK 0.0611
PK tK 0.0611
PK1 tK1 0.0611
vision deep 0.0611
SSS Tuned GSCNN 0.5231
BharatAI BharatAI-t2 0.4933
PV=NRT segmantation_model 0.0382
coe deep trying 0.5296
NAS_IITRPR Baseline 0.0003
Ryan Dsouza Spatial pooling pyramid 0.0162
NAS_IITRPR Baseline 0.4435
Ryan Dsouza Gods Touch 0.0212
Ryan Dsouza Gods Touches 0.0445
rajat modi tbd 0.4348
godzilla godzilla 0.4348
godzilla2 godzilla2 0.4354
SSS Resnet-101 0.5676
Ryan Dsouza low score 0.0665
PPR Deep Method 0.0729
PPR Deep Method 0.5611
PV Seg_model 0.0403
USSB Try2 0.5505
SSN_CSE UNet 0.0778
SSN_CSE FCN 1 0.4727
SSN_CSE UNet Modified 0.4705
godzilla3 mo 0.4329
KLE Tech_Team_02 VAENet 0.5232
KLE Tech_Team_02 VAENet 0.5196
godzilla4 godzilla4 0.4433
Himanshu Mittal SemanticSegmentation_Enet_v3 0.0
godzilla5 godzilla5 0.4374
godzilla5 godzilla5 0.4374
godzilla6 godzilla6 0.4373
SGGS-IITJ Deep CNN 0.5588
USSB Try3 0.3388
PARIMAL PARIMAL 0.528
PPR Deep method1 0.5681
Tabasco Trial 0.5438
mohit_asudani resnet 0.4243
Alphamales Baseline 0.2405