Milestone I: Team proposals

• Team up with a friend. Please use ODTUclass forum if needed.
• You are expected to upload your team information on ODTUclass.
• Each project group will consist of 2 students. If you cannot form a pair, I'll do random pairing. All groups will consist of pairs. We might have one exceptional 3-member or 1-member group, depending on the active number of students taking the class.
• Please email me asap if you ever decide to practically "drop" the class.
• Please email me if you could not form a group by the deadline so that I can do that I can quickly do random pairing.
• Start working on your project proposals (next section) immediately when you form your project group.

ODTUclass submission format.

Exactly one submission (from a single member) of each project group is expected. Each group should submit one text file with the following content:

StudentID_1 Name_1
StudentID_2 Name_2

Students who end up with no groups should submit NO_GROUP, and email me for notification.

Milestone II: Project proposals

• Look for generative model papers published at a top venue or journal (primarily, CVPR, ECCV, ICCV, ICML, ICLR, NeurIPS, IJCV, JMLR, TPAMI) recently (ideally published within the past 18 months). Focus on papers directly relevant for this class. Exceptionally good arXiv preprints can also be considered.
• Papers proposing new unsupervised generative model formulations are particularly encouraged. The model does not have to be for the image domain, other domains are also accepted.
• Application-only papers are discouraged. However, papers addressing other directly relevant topics, such as conditional generative models (e.g., class / text conditional), image to image translation models, etc. can be considered unless they are purely application oriented. Again: papers innovating in terms of core generative model formulation are strongly preferred.
• Note that you will be expected to reproduce some of the experimental results of the paper to validate your implementation. Therefore, make sure you do not choose papers for which your compute resources are insufficient. While it is possible to reproduce results on smaller-scale datasets like CIFAR-10/100 in most cases (on free resources such as Google colab / on CPU / etc), reproducing results on high-resolution ImageNet or video datasets can be computationally very demanding. Therefore, make sure you choose papers reporting results on benchmarks feasible for your resources. (You will not be required to reproduce other large-scale experimental results of the paper.)
• A group's project paper needs to be distinct from the papers presented by group members. It is OK to work on a paper presented by a member of another group.

Policy on code and data "reuse".

• If a publicly available implementation of the paper exists, you are forbidden to make use of it. The paper should be your only resource, you should not be reusing or re-purposing existing source model implementations. Projects violating this policy may receive as low as zero-grade. Exceptionally, I may allow looking into existing code where a part of paper is obviously unclear, but you must ask for permission.
• If your implementation requires a large pretrained feature extraction model (eg. ImageNet pre-trained Inception/VGG/ResNet/etc), you may use an existing pre-trained model.
• If you need to use a data driven evaluation metric (eg. FID, Inception Score, etc), you are supposed to implement it yourself. As written in (1), you may use existing pre-trained networks typically required for implementing them.
• It is fine to download and use existing publicly-available datasets. This can be raw image/sound data or this can be a dataset of pre-extracted features (eg. VGG) if the model is to be trained and evaluated in the feature space. However, you are normally not allowed to download and use existing data that is supposed to be obtained by some part of the model that you are implementing.
• Ask me whenever you are in doubt.

ODTUclass submission format. Propose 5 paper titles in higher-to-lower preference order as a simple text file, in the following format:

• First line: HIGHEST preference.
• Last line: LOWEST preference.
• There should be 5 lines in total (one extra empty line at the very end is not a problem).
• Each line should contain the exact title of the original publication. If the same work appears also on arxiv, please use the paper title published on the conference website instead.
• Avoid quotation marks (" and ').

The submissions will be processed as follows:

• I will filter out papers that looks unsuitable.
• In case of conflicts, I will assign titles to groups by using their preference order.
• In cases of ties, I may contact the groups.
• In case none of the proposals of a group are accepted (due to higher-ranking preferences by other groups or unsuitability of the titles), new proposals will be requested.

Milestone III: Declare experimental result goals

• This will be the first milestone after the announcement of group - paper matches.
• Each group is expected to declare one qualitative (eg. image samples) and one quantitative result (eg. FID/IS/etc scores on one dataset) that will be targeted to reproduce. These results should be comparable to those reported in the paper, as a validation of your implementation. In specific cases, you may need to produce more than one result, if that is necessary to validate your implementation.
• Some papers report results only at large-scale datasets / very high resolution images. Such papers can be difficult to reproduce. This milestone exists to “force” everyone to declare their experimental result goals, ie. decide while qualitative and quantitative results that you're going to reproduce. In this way, you can both clarify your goals and realize any potential problems with your paper choice before it is too late.
• I also understand that many of the interesting recent works report high-resolution results only. If this is the case for your project, you may also consider proposing a lower resolution setting, which can greatly cut the compute resources needed. In such a case, you need to pay attention to the following in declaring your experimental goals:
  • discuss the problem (ie. what experiments exist in the paper, and why reproducing all those results look problematic), and indicate whether your proposed lower-resolution setting is your main target or just a fall-back plan,
  • clearly indicate your target resolution, and explain why it will still be sufficient to demonstrate the abilities of the model (you may want to visually explore sample images),
  • explain how you'll compare your quantitative results with those in the paper (you may adapt/propose a heuristic to make the numbers across scales roughly comparable).
• Normally you should choose your paper carefully in the first step. But if you realize new major problems with your paper choice at this stage (eg. the paper completely lacks quantitative results, etc), please contact me immediately, explaining your problem. In the worst case, I may ask you to propose a different paper, but please note that you’ll loose time for making such a change at this late stage.
• What you declare will be your minimum goals. You may definitely optionally produce additional results, as an enhancement of your open source code.
• Important note I. Please note that failing to reproduce results is not necessarily penalized heavily, as long as the group presents and reports (as part of the project) the detailed and creative attempts and engineering efforts to fix the encountered problems. Still, there is likely to be some (minor or major) penalty in case of a large discrepancy between project results and paper results, as an incentive to try hard to reproduce the papers. Similarly, magically / mysteriously obtained good results do not necessarily result in high project grades, especially if the group fails to show the engineering process or fails to show a good grasp of the method / their own implementation at the end of the project.
• Important note II. Reproducing results at relatively small (but not exteremely small) resolutions is much better than obtaining no good results at all.

ODTUClass submission format: Upload a simple text file, declaring which minimum set of results do you plan to re-produce. In this file, give the title and URL of your project paper, and point to table(s) (you may point to specific rows) and figure(s) that you aim to reproduce.

Milestone IV: Project "Version 1" submissions

• This is supposed to be your complete and polished submission involving your own implementation of the paper you have selected in PyTorch and your experimental results.
• Submission format and additional guidelines will be announced separately. Please also check the peer-review form (see below) to better understand the expectations.
• As explained in the first lecture and the course homepage, I will grade this together with the second version at the end of the semester, as part of your project grade.
• Note that this deadline is close to the midterm date, therefore, please plan your work ahead.
• The software requirements for Version 1 (and the submission details) are given below.

Milestone V: Peer-review report submissions

• Each group will review one other project and will give written feedback, which is expected to help finding and fixing bugs in the reviewed project. Reviews themselves will be graded by me.
• This form will be used for preparing the peer-review reports: Peer review form.

ODTUClass submission format: Convert your review form into pdf, and name it as ReviewedGroup##_ByGroup##.pdf.

Milestone VI: Project "Version 2" submissions

• This final version of the projects will be made public on a joint Github repository, where each project submission will be a sub-component of the repository. Source code authors and reviewers of each component will be clearly denoted in the repository.
• All projects are planned to be licensed with the permissive MIT License for its simplicity. Please contact me immediately (by the week 11) if you have any objections or thoughts about this, I welcome feedback and will be happy to discuss other options. If I do not receive an objection, you will be presumed to accept the license that your submission will receive.
• The software requirements for Version 2 (and the submission details) are given below.

Milestone VII: Project demos

Each group will demonstrate its project. The rules and guidelines:

• Fully complete your project before the presentation.
• Prepare your jupyter notebooks and be-ready to show your projects (possibly using screen sharing).
• Each presentation should be around 8 (max 10 mins) minutes-long in total. Rehearse accordingly.
• Explain the key ideas and key equations of the paper that you've implemented, using the summary that you have written into the jupyter notebook.
• Explain your experimental result goals.
• Show your own results (qualitatively and quantitatively) and explain whether you have reached your experimental result goals. If not, summarize your efforts and your ideas on what's going on (please try to be more precise than just "wrong hyper-parameters").
• Quickly go over your code (show jupyter notebook and other python files), briefly summarize your implementation.
• Tell us about the difficulties that you have encountered in your project (make sure these are also summarized in the jupyter notebook).