Bengali Visual Genome 1.0 Release Description
-----------------------------------------------
http://hdl.handle.net/11234/1-3722

Authors:
  Arghyadeep Sen, Shantipriya Parida, Ketan Kotwal, Subhadarshi Panda,
  Ondřej Bojar, and Satya Ranjan Dash.

Data
----
Bengali Visual Genome (BVG for short) 1.0 has similar goals as Hindi Visual
Genome (HVG) 1.1: to support the Bengali language. Bengali Visual Genome 1.0 is
the multi-modal dataset in Bengali for machine translation and image
captioning. Bengali Visual Genome is a multimodal dataset consisting of text
and images suitable for English-to-Bengali multimodal machine translation tasks
and multimodal research. We follow the same selection of short English segments
(captions) and the associated images from Visual Genome as HGV 1.1 has. For
BVG, we manually translated these captions from English to Bengali taking the
associated images into account. The manual translation is performed by the
native Bengali speakers without referring to any machine translation system.

The training set contains 29K segments. Further 1K and 1.6K segments are
provided in development and test sets, respectively, which follow the same
(random) sampling from the original Hindi Visual Genome. A third test set is
called the ``challenge test set'' and consists of 1.4K segments. The challenge
test set was created for the WAT2019 multi-modal task by searching for
(particularly) ambiguous English words based on the embedding similarity and
manually selecting those where the image helps to resolve the ambiguity. The
surrounding words in the sentence however also often include sufficient cues to
identify the correct meaning of the ambiguous word.

Dataset Formats
---------------
The multimodal dataset contains both text and images.

The text parts of the dataset (train and test sets) are in simple tab-delimited
plain text files.

All the text files have seven columns as follows:

Column1 - image_id
Column2 - X
Column3 - Y
Column4 - Width
Column5 - Height
Column6 - English Text
Column7 - Bengali Text

The image part contains the full images with the corresponding image_id as the
file name. The X, Y, Width and Height columns indicate the rectangular region
in the image described by the caption.


Data Statistics
---------------
The statistics of the current release are given below.

Parallel Corpus Statistics
--------------------------

Dataset                   Segments    English Words            Bengali Words
----------                --------    -------------            -------------
Train                       28930      143115                  113978
Dev                           998        4922                    3936
Test                         1595        7853                    6408
Challenge Test               1400        8186                    6657
----------                --------    -------------            -------------
Total                       32923      164076                  130979

The word counts are approximate, prior to tokenization.

Citation
--------
If you use this corpus, please cite the following paper:

@inproceedings{hindi-visual-genome:2022,
  title= "{Bengali Visual Genome: A Multimodal Dataset for Machine Translation and Image Captioning}",
  author={Sen, Arghyadeep
      and Parida, Shantipriya
      and Kotwal, Ketan
      and Panda, Subhadarshi
      and Bojar, Ond{\v{r}}ej
      and Dash, Satya Ranjan},
  editor={Satapathy, Suresh Chandra
      and Peer, Peter
      and Tang, Jinshan
      and Bhateja, Vikrant
      and Ghosh, Anumoy},
  booktitle= {Intelligent Data Engineering and Analytics},
  publisher= {Springer Nature Singapore},
  address= {Singapore},
  pages = {63--70},
  isbn = {978-981-16-6624-7},
  doi = {10.1007/978-981-16-6624-7_7},
}