This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
public:gsoc:projectnephos [2018/02/13 19:17]
public:gsoc:projectnephos [2018/02/17 15:19]
Line 5: Line 5:
 **Project Nephos: Cloud based storage for a massive collection of TV recordings** **Project Nephos: Cloud based storage for a massive collection of TV recordings**
-Both CCExtractor and Red Hen (our sister organization you should ​check out) store massive archives of TV recordings. By massive we mean hundreds of terabytes. Until now these archives have been handled in-house, but we're approaching a point in which it's financially more sensible to use cloud storage.+There'​s a lot of documentation on our close friend organization ​Red Hen (just Google them, or check out their ideas page), but for our purposes these are the basic ideas:
-During this summer we want to approach ​the migration ​to cloud storagespecifically using Google Drive (but your code should allow to extend ​to other services). +- Red Hen is an informal (as in they'​re tied by collaboration,​ not contracts) group of entities, most of them large universities,​ from many places around ​the world that share resources. These resources are hardware, software, media, knowledge, source code, people, and access ​to people.\\ 
 +- Everything is openthere'​s no NDAs in place, or proprietary software or anything like that. The one thing is that limited ​(to researchers and other people that can benefit from itis access to the media repository due to concerns on copyright.\\ 
 +- One of the things some universities do (and most want to do) is record as many local (to them) TV channels as they can and archive them. They are used for a long list of research and analysis topics, from language trends to body language analysis, to catch politics on lies and almost anything you can think of.\\ 
 +- They use CCExtractor to generate transcripts of the media files. This is the original link between Red Hen and us, but since we met we have been collaborating on other overlapping interests.\\
-Some of the must-have features are easy, for example when a recording is complete (and exists as a local file) is needs to be moved to Cloud. ​+As mentioned, some universities record a large number of TV channels available locally to them, so UCLA records what they can get in Los Angeles, UNav what they can get in Navarra, Spain, and so on. Currently storage is either handled in-house, or it is uploaded to UCLA, where it is stored. As pointed out, this is all done on good will, not contract, but this is a bit besides the point.  
 +We've reached a point in which storing these files locally is making less and less sense. Google offers unlimited storage for organizations that use Google Apps (their professional cloud suite, with GMail, Drive, and so on) which some universities such as UNav have.  
 +The project during this summer is migrate to cloud storage, which will require creating a number of tools, modifying others, figuring out the best way to handle access permissions to the files in the cloud, general organization and so on. 
 +Some of the must-have features are easy, for example when a recording is complete (and exists as a local file) it needs to be moved to Cloud. ​
 Other things will need more work. Specifically:​ Other things will need more work. Specifically:​
  • public/gsoc/projectnephos.txt
  • Last modified: 2018/03/07 19:46
  • by cfsmp3