NCVPRIPG 2019 IDD Benchmark

Semantic segmentation on IDD Lite Dataset

This challenge is open to students in Indian universities and colleges. The challenge will use IDD-Lite, a dataset that is small and compact to fit on any personal computer and so will not huge-compute infrastructure. IDD-Lite is less than 50MB is size, contains 7 classes (compared to 30 in IDD).

Directions for Participation

  1. Register here for receiving updates about the competition.
  2. Register an account at the IDD website: http://idd.insaan.iiit.ac.in/
  3. After logging in to IDD website, download the IDD Lite dataset (http://idd.insaan.iiit.ac.in/dataset/download/)
  4. Segmentations masks have all been already generated for IDD Lite in the download.
  5. Once you have built a model, and have the predictions of the model in any of the split (train, val), you can evaluate the metric as directed here: https://github.com/AutoNUE/public-code#evaluation. Use the following command for segmentation evaluation:
    python evaluate/idd_lite_evaluate_mIoU.py --gts $GT --preds $PRED --num-workers $C
    Your predictions is a png image, which has the size of 256x128. Each pixel of this image contains the label as level1Ids (see labels code) of the corresponding image (resized to 256x128). The evaluation code above resizes both your prediction and ground truth png files to 256x128, in case they are not of that size.
  6. Finally you can upload the predictions for the test split, to be evaluated for the leaderboard here: http://idd.insaan.iiit.ac.in/evaluation/submission/submit/
Results
Team/Uploader Name Method Name mIoUmIoU for L1 IDs at 128p
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.0
Swetha B Enet 0.3986
Swetha B Enet 0.3986
mohit_asudani resnet 0.3885
Testing Testing 0.0959
testing testing 0.0959
tensorcat dlv3p 0.0031
mohit_asudani resnet 0.3885
kt001 trial 0.0006
bekar_phd_student bekar 0.4616
USSB Try1 0.5614
SSN_CSE UNet 0.1361
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.2271
KLE Tech_Srikar CustomNet 0.0321
tyrion first_try 0.4043
Sanchayan Santra nnet 0.3235
SSN_CSE FCN 0.0926
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.4738
kt002 trial 0.0321
SGGS-IITJ testing 0.5176
First_Resnet Residual U-Net 0.5838
bekar_phd_students Unemployment ki shakti 0.4968
SGGS-IITJ Server 1 0.5588
SGGS-IITJ Deep CNN 0.5586
bekar_phd_students Unemployment ki shakti 0.4809
test test 0.3235
Residual U-Net Residual U-Net2 0.5935
TCE RS Lab Sematic Segmentation U-net 0.532
SSN_CSE Modified FCN 0.3438
Deepsegmentation Deep Encoder Decoder Network for semantic segmetnation 0.5017
SSN_CSE FCN 2 0.3771
KLE Tech_Team CustomNet 0.538
KLE Tech_Team CustomNet 0.4563
TCE RS Lab Sematic Segmentation U-net 0.5323
bekar_phd_students Unemployment ki shakti 0.5293
KLE Tech_Team CustomNet 0.0781
KLE Tech_Team CustomNet 0.0768
Parimal TBd 0.0611
SSS Gated Shape CNN 0.4872
kz001 CNet 0.5232
kz002 CNet 0.5196
Deepsegmentation Deep Encoder Decoder Network for semantic segmentation 0.5017
BharatAI BharatAI-t1 0.5262
TCE RS Lab Sematic Segmentation U-net 0.5432
kz003 CNet 0.4802
SSS Tuned GSCNN 0.5231
SGGS-IITJ testing 0.5296
SSN_CSE FCN 3 0.3137
PK tK 0.0611
test test 0.4748
SSN_CSE FCN 4 0.3407
t1 t2 0.4831
kz004 CNet 0.4349
Parimal TBd 0.0611
PK tK 0.0611
PK1 tK1 0.0611
vision deep 0.0611
PV=NRT segmentation_model 0.0382
BharatAI BharatAI-t2 0.4933
NAS_IITRPR Baseline 0.0003
coe deep trying 0.5296
Ryan Dsouza Spatial pooling pyramid 0.0162
NAS_IITRPR Baseline 0.4435
Ryan Dsouza Gods Touch 0.0212
Ryan Dsouza Gods Touches 0.0445
rajat modi tbd 0.4348
godzilla godzilla 0.4348
godzilla4 godzilla4 0.4433
godzilla2 godzilla2 0.4354
mohit_asudani resnet 0.4243
SSS Resnet-101 0.5676
Ryan Dsouza low score 0.0665
PPR Deep Method 0.0729
Himanshu Mittal SemanticSegmentation_Enet_v3 0.0
PPR Deep Method 0.5611
PV Seg_model 0.0403
USSB Try2 0.5505
godzilla3 mo 0.4329
SSN_CSE UNet 0.0778
SSN_CSE FCN 1 0.4727
SGGS-IITJ Deep CNN 0.5588
SSN_CSE UNet Modified 0.4705
KLE Tech_Team_02 VAENet 0.5232
godzilla5 godzilla5 0.4374
KLE Tech_Team_02 VAENet 0.5196
PPR Deep method1 0.5681
godzilla5 godzilla5 0.4374
USSB Try3 0.3388
godzilla6 godzilla6 0.4373
PARIMAL PARIMAL 0.528
Tabasco Trial 0.5438
Alphamales Baseline 0.2405
tabasco Trial2 0.5511
USSB DCNN 0.5502
SGGS-IITJ Try_Server2 0.5201
USSB pur7 0.6009
mohit_asudani resnet 0.4516
Deepsegmentation Deep Encoder Decoder Network for semantic segmetnation 0.306
SGGS-IITJ RESNET 0.5391
Ryan Dsouza hallelujah 0.0403
USSB dxb13 0.507
coe deep custom_net 0.4845
SSS Tuned GSCNN 0.5926
jai mata ki jai mata ki 0.4777
SSS Tuned GSCNN 0.5785
mohit_asudani resnet 0.4515
LYNX DRNet 0.0693
USSB DCNN 0.5519
TCE RS Lab Sematic Segmentation U-net 0.5586
USSB DCNN 0.5193
kz005 CNet 0.3092
USSB pur9 0.5926
PARIMAL U Net 0.5807
Lynx DRNet2 0.0714
lab lab 0.5246
Lynx DSP2 0.0623
Lynx DSP 0.0632
Ryan Dsouza Devils dice 0.3117
Residual U-Net Custom Net 0.6081
TCE RS Lab Sematic Segmentation Residual U-net 0.5838
parimal_1 secret 0.4183
Cerebro PSPnet 0.0779
PARIMAL U Net 0.5841
Residual U-Net Custom Net 0.6238
SSS Tuned GSCNN 0.5832
Aayush encoder_decoder 0.3147
SSS Tuned GSCNN 0.6014
onemoretry onemoretry 0.4251
SGGS-IITJ Dilated J-ResNet 0.5661
USSB pirv2 0.611
CV Lab Deep ConvNet 0.1921
SGGS-IITJ DIl_r_resnet 0.5661
Aayush encoder_decoder 0.2509
Cerebro Resnet50 0.068
Cerebro Resnet50 0.0691
USSB DCNN 0.5733
NAS_IITRPR Baseline 2 0.536
Aayush encoder_decoder 0.2875
USSB irv2 0.611
SGGS-IITJ RESNET 0.5384
tabasco OCR 0.5338
Aayush encoder_decoder 0.2271
SSS Tuned GSCNN 0.6141
Aayush encoder_decoder 0.2271
Aayush encoder_decoder 0.3147
NAS_IITRPR Meta Baseline 0.5336
tensorcat dlv3p 0.5511
PARIMAL U Net + CRF 0.5397
SGGS-IITJ Dil_r_resnet(tuned) 0.564
Himanshu Mittal MEnet 0.0403
BharatAI BharatAI-t0 0.5035
Himanshu Mittal MEnetv1 0.0403
UNET UNET 0.5369
BharatAI t3 0.5344
CV Lab Deep ConvNet 0.1919
CV Lab Deep ConvNet 0.4875
USSB tuned pur7 0.6009
Espada Ensemble 0.531
USSB tuned pirv2 0.613
USSB e7net 0.6175
NAS_IITRPR Model 2 0.5386
Espada ENET 0.4705
USSB e5net 0.6087
PARIMAL Tuned U-Net 0.4654
BharatAI t4 0.5748
Espada Ensemble 0.531
BharatAI BharatAI-t2 0.5588
BharatAI Morphological Network 0.5823
Espada UNET 0.5729
PHJA lnet 0.5276
lab lab 0.5714
BharatAI t4 0.5699
SGGS-IITJ custom unet 0.6039
tyrion trying 0.4333
SGGS-IITJ custom-unet 0.6166
BharatAI t4 0.5838
lab lab 0.5519
PARIMAL Tuned Tiramisu 0.5965
small s1 0.5897
SGGS-IITJ Tuned Custom U-net 0.5515
CV Lab Deep ConvNet 0.4949
SSS Tuned GSCNN 0.2229
USSB ppirv2 0.6175
SGGS-IITJ Tuned Custom U-Net 0.5515
small s1 0.5788
CV Lab SegNet 0.4884
USSB p eff7net 0.6276
SGGS-IITJ Custom Net_final 0.6117
Residual U-Net Custom Net 0.6245
small s1 0.5897
Espada UNET 0.5577
SSS Tuned GSCNN 0.6167
bekar_phd_students Unemployment ki shakti 0.5718
PARIMAL Tuned Tiramisu 0.5952
SSS Tuned GSCNN 0.2849
Aayush encoder_decoder 0.3451
BharatAI Exp-Net 0.5897