Commit 246339aa authored by Matej Choma's avatar Matej Choma

milestone commit

parent fc90d1e5
\ No newline at end of file
# NI-MVI-sp-2020-chomamat # Upscaling of weather radar video resolution
The repository for NI-MVI semestral work on video resolution upscaling. The repository for NI-MVI semestral work on video resolution upscaling.
## Assignment ### Assignment
Vezměte krátké 5-10 sekundové video a vytvořte generátor, který bude generovat Vezměte krátké 5-10 sekundové video a vytvořte generátor, který bude generovat
video s vyšším rozlišením. Používejte obě architektury: GAN a U-Net. video s vyšším rozlišením. Používejte obě architektury: GAN a U-Net.
Porovnejte výsledky. Porovnejte výsledky.
## Milestone
The objective of this work is to 4x upscale the resolution of `data/target.mp4` video. The video contains 24 hours of weather radar precipitation data with a resolution of 480x270 pixels. Weather radar data is generally noisy, which poses a secondary challenge of denoising the data during SR.
![Target Video](data/target.mp4)
### Planned approach to the semestral work
The upscaling of video resolution can be decomposed into upscaling of individual frames. Thus, I will focus on image super-resolution (SR) ML models and use the best performing one to generate target video.
I plan to build and train the following models:
* U-net, initially motivated by my bachelor's thesis [1]. From my experience, this architecture is able to pick the low hanging fruits in various computer vision tasks. The U-net described in [2] won second place at NTIRE2019 challenge [4], which supports this claim. I plan to utilize the findings from [2] for the training of the U-net.
* SRGAN [5], which is the first utilization of the GAN framework for the SR task.
I will use the DIV2K dataset [6] for training. For validation, I will use the Set14 benchmark [7], and weather radar validation set created for this work. I will use both qualitative evaluation and quantitative evaluation with PSNR and SSIM metrics.
![weather radar image](data/examples/radar.png "1920x1080 weather radar image")
I am posing the following questions:
* Can SR model trained on camera images generate weather radar images?
* Does evaluation on Set14 benchmark correlate with the weather radar validation set?
* Can training or finetuning on weather radar data improve the performance?
>All of the weather radar data was provided by the Czech company [Meteopress](
### Literature
* [[1]]( Choma, Matej. *Interpolation and Extrapolation of Subsequent Weather Radar Images.*
* [[2]]( Feng, Ruicheng, et al. *Suppressing model overfitting for image super-resolution networks.*
* [[3]]( Lugmayr, Andreas, Martin Danelljan, and Radu Timofte. *Ntire 2020 challenge on real-world image super-resolution: Methods and results.*
* [[4]]( Cai, Jianrui, et al. *Ntire 2019 challenge on real image super-resolution: Methods and results.*
* [[5]]( Ledig, Christian, et al. *Photo-realistic single image super-resolution using a generative adversarial network.*
* [[6]]( Agustsson, Eirikur and Timofte, Radu. *NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study.*
* [[7]]( Zeyde, Roman, Michael Elad, and Matan Protter. *On single image scale-up using sparse-representations.*
import math
import matplotlib.pyplot as plt
import numpy as np
def _squeeze(x):
_x = np.squeeze(x)
while _x.ndim < 3:
_x = _x[None, ...]
return _x
def grid(X, cols=None, size=(15, 10), title=None, vmax=None,
x = _squeeze(np.array(X))
rows = math.ceil(len(x) / cols)
fig, axes = plt.subplots(nrows=rows, ncols=cols,
figsize=(cols*size[0], rows*size[1]),
sharex=True, sharey=True)
ax = axes.ravel()
for i in range(rows):
for j in range(cols):
if i*cols + j < len(x):
ax[i*cols + j].imshow(x[i*cols+j], cmap=cmap, vmax=vmax)
if title is not None:
fig.suptitle(title, fontsize=30)
fig.tight_layout(rect=[0, 0.03, 1, 0.93])
def show(x, size=(15, 10), title=None, vmax=None,, **kwargs):
if title is not None:
if (x.ndim == 2):
return plt.imshow(x, cmap=cmap, vmax=vmax, **kwargs)
return plt.imshow(x, **kwargs)
\ No newline at end of file
File added
This diff is collapsed.
%% LaTeX template class for technical report
%% for subject MVI in Czech Technical University in Prague (CTU)
%% (c) 2008 Vit Zyka
%% History:
%% 0.1 2008-09-21 new for NRP
%% 0.2 2008-11-12 href for email, unicode in hyperref; tolerance; microtype; cmap
%% \texorpdfstring
%% 0.3 2018-02-12 update for purposes of MI-MVI
\typeout{Class mvi-report v0.3, 2018-02-12, Martin Slapak, Vit Zyka}
%% Options...
%% Dimensions...
%% Packages...
\RequirePackage[pdfpagemode=UseNone,pdfborder=0 0 0,unicode]{hyperref}
%% Title...
\newtoks\MVIemail \def\email#1{\MVIemail={#1}}
\newtoks\MVIaffiliation \def\affiliation#1{\MVIaffiliation={#1}}
% pokus o usetreni mista v \paragraph
{paragraph} % the name
{4} % the level
{\z@} % the indent
{1.5ex \@plus1ex \@minus.2ex} % the before skip
{-1em} % the after skip
{\normalfont\normalsize\bfseries} % the style
%% Bibliography...
% citace dle abecedy
% citace dle poradi, ale nevejde se to
% citace dle poradi, ale vejde se to ;-)
%% Misc...
% pokud nechcem aby nam to zmensovalo pismenka v nazvech titulu,
% tak to obklopit chlupatejma zavorkama
@misc{ zizka,
title = {Some interesting source},
author = { Jan {\v Z}i{\v z}ka},
year = {1415},
howpublished = {online},
note = {[cit.~2018--10--11] \url{http://zizka.trocnov/husiti/conspiration.pdf} }
\@writefile{toc}{\contentsline {section}{\numberline {1}\IeC {\'U}vod}{1}{section.1}}
\@writefile{toc}{\contentsline {section}{\numberline {2}Vstupn\IeC {\'\i } data}{1}{section.2}}
\@writefile{toc}{\contentsline {section}{\numberline {3}Metody}{1}{section.3}}
\@writefile{toc}{\contentsline {section}{\numberline {4}V\IeC {\'y}sledky}{1}{section.4}}
\@writefile{lof}{\contentsline {figure}{\numberline {1}{\ignorespaces Vliv parametru \emph {Y}}}{1}{figure.1}}
\newlabel{fig:par-y}{{1}{1}{Vliv parametru \emph {Y}}{figure.1}{}}
\@writefile{lof}{\contentsline {figure}{\numberline {2}{\ignorespaces Vliv parametru \emph {X}}}{1}{figure.2}}
\newlabel{fig:par-x}{{2}{1}{Vliv parametru \emph {X}}{figure.2}{}}
\@writefile{toc}{\contentsline {section}{\numberline {5}Z\IeC {\'a}v\IeC {\v e}r}{1}{section.5}}
This diff is collapsed.
\BOOKMARK [1][-]{section.1}{\376\377\000\332\000v\000o\000d}{}% 1
\BOOKMARK [1][-]{section.2}{\376\377\000V\000s\000t\000u\000p\000n\000\355\000\040\000d\000a\000t\000a}{}% 2
\BOOKMARK [1][-]{section.3}{\376\377\000M\000e\000t\000o\000d\000y}{}% 3
\BOOKMARK [1][-]{section.4}{\376\377\000V\000\375\000s\000l\000e\000d\000k\000y}{}% 4
\BOOKMARK [1][-]{section.5}{\376\377\000Z\000\341\000v\001\033\000r}{}% 5
%% Created in 2018 by Martin Slapak
%% Based on file for NRP report LaTeX class by Vit Zyka (2008)
%% Compilation:
%% >pdflatex report
%% >bibtex report
%% >pdflatex report
%% >pdflatex report
\title{Precipitation Video Resolution Upscaling}
\author{Matej Choma}
\affiliation{ČVUT - FIT}
Definice problému/úkolu, který práce řeší\ldots
\section{Vstupní data}
Původ, proces získání, předzpracování, \ldots
Použité metody, jejich přizpůsobení, aplikace\ldots
Jakých výsledků bylo dosaženo, co na ně melo vliv. Srovnání s očekáváním, \emph{diskuze nad výsledky} -- zvláště důležitá v případě, že něco vyšlo \emph{divně}.
\caption{Vliv parametru \emph{Y}}
\caption{Vliv parametru \emph{X}}
Jak je z ilustrací \ref{fig:par-y} a \ref{fig:par-x} patrné, není to totéž, protože\ldots
% --- ZAVER
K čemu to bylo/je dobré, jak to půjde využít dále, co by šlo ještě vylepšit\ldots
% --- Bibliography
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment