Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision Both sides next revision
public:gsoc:projectnephos [2018/02/13 19:17]
cfsmp3
public:gsoc:projectnephos [2018/02/16 18:41]
cfsmp3
Line 5: Line 5:
 **Project Nephos: Cloud based storage for a massive collection of TV recordings** **Project Nephos: Cloud based storage for a massive collection of TV recordings**
  
-Both CCExtractor and Red Hen (our sister organization you should ​check out) store massive archives of TV recordings. By massive we mean hundreds of terabytes. Until now these archives have been handled in-house, but we're approaching a point in which it's financially more sensible to use cloud storage.+There'​s a lot of documentation on our close friend organization ​Red Hen (just Google them, or check out their ideas page), but for our purposes these are the basic ideas:
  
-During this summer we want to approach ​the migration ​to cloud storage, ​specifically using Google ​Drive (but your code should allow to extend ​to other services)+- Red Hen is an informal (as in they'​re tied by collaboration,​ not contracts) group of entities, most of them large universities,​ from many places around the world that share resources. These resources are hardware, software, media, knowledge, source code, people, and access to people.\\ 
 +- Everything is open, there'​s no NDAs in place, or proprietary software or anything like that. The one thing is that limited (to researchers and other people that can benefit from it) is access to the media repository due to concerns on copyright.\\ 
 +- One of the things some universities do (and most want to do) is record as many local (to them) TV channels as they can and archive them. They are used for a long list of research and analysis topics, from language trends to body language analysis, to catch politics on lies and almost anything you can think of.\\ 
 +- They use CCExtractor to generate transcripts of the media files. This is the original link between Red Hen and us, but since we met we have been collaborating on other overlapping interests.\\ 
 + 
 +As mentioned, some universities record a large number of TV channels available locally ​to them, so UCLA records what they can get in Los Angeles, UNav what they can get in Navarra, Spain, and so on. Currently ​storage ​is either handled in-houseor it is uploaded to UCLA, where it is stored. As pointed out, this is all done on good will, not contract, but this is a bit besides the point.  
 + 
 +We've reached a point in which storing these files locally is making less and less sense. ​Google ​offers unlimited storage for organizations that use Google Apps (their professional cloud suite, with GMail, Drive, and so on) which some universities such as UNav have.  
 + 
 +The project during this summer is migrate ​to cloud storage, which will require creating a number of tools, modifying others, figuring out the best way to handle access permissions to the files in the cloud, general organization and so on.
  
 Some of the must-have features are easy, for example when a recording is complete (and exists as a local file) is needs to be moved to Cloud. ​ Some of the must-have features are easy, for example when a recording is complete (and exists as a local file) is needs to be moved to Cloud. ​
  • public/gsoc/projectnephos.txt
  • Last modified: 2018/03/07 19:46
  • by cfsmp3