{"id":11218,"date":"2016-06-21T13:50:49","date_gmt":"2016-06-21T11:50:49","guid":{"rendered":"https:\/\/oewf.org\/en\/?p=11218"},"modified":"2016-06-22T10:17:19","modified_gmt":"2016-06-22T08:17:19","slug":"internship-2016-terrain-estimation-monocular-camera","status":"publish","type":"post","link":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/","title":{"rendered":"Internship 2016: Terrain Estimation by a Monocular Camera"},"content":{"rendered":"

It\u2019s been one month here and I have learnt tons of things working. I wouldn\u2019t say everything is going according to the schedule, but I believe research projects are like this, we hope that something will work and go further but end up not using it because of some constraints and try to find another way out.<\/p>\n

Project Overview<\/b><\/p>\n

A Rover or an astronaut is mounted with a monocular camera from which the live streaming data can be accessed through a ROS topic. At every step, the camera can view some part of the region which is in front of it but not the terrain which is exactly below or immediate to it. So, visual odometry is used to find the position of the camera from which we can obtain the patch of the terrain which is immediate to the rover or an astronaut from the frames which are processed before the rover or astronaut reached that particular position.<\/p>\n

We know the world coordinates of the camera from Visual Odometry and since we have calibrated the camera, we have all the parameters to get the image points on previous frames of corresponding world points. So, for every world point of the camera trajectory, we can get an approximated world point of the astronauts\u2019 foot or wheels of the rover. We check if the particular world point is visible in the previous frames and if visible, we collect the patches of terrain from all the frames in which the corresponding world point is visible.<\/p>\n

For now, it\u2019s assumed that the astronaut gives the traversability of terrain on which he is standing by a push button and that this, with the patch that we obtained, can be given to a regression model as a training data. We should consider getting the labels for the terrain patches automatically by an accelerometer. The main thing that should be noticed is that we aren\u2019t classifying the terrains but trying to predict the traversability for the given patch of the terrain.<\/p>\n

So, in the beginning, the algorithm is not expected to give good results but in considerable time, it is assumed to give better estimation of traversability. The further ideas would include considering the slope of the terrain which is an important aspect of traversability.<\/p>\n

Visual Odometry<\/b><\/p>\n

Initial attempts were made on Semi-Direct Visual Odometry (SVO). It worked fine on the test data but didn\u2019t deliver good results on a live streaming camera. The algorithm is tested with Microsoft Lifecam HD, Logitech and Pointgrey BFLY PGE cameras with focus set to infinity, constant exposure and brightness. Even after trying with different values of FPS, the algorithm didn\u2019t deliver optimum results. SVO is considered good for downward looking camera and proves to produce robust results for MAV\u2019s but not for a forward looking camera like in our case.<\/p>\n

The next attempt is made on Ethzasl Ptam. It gave optimum results for even a forward looking camera. Considering the computational power we have on the spacesuit, we may consider running the mapping thread of the Visual Odometry on a ground station which would save a lot of computational time on spacesuit.<\/p>\n

\"Stereo<\/a>

Stereo Initialization (c) OeWF<\/p><\/div>\n

\"Feature<\/a>

Feature tracking and AR grid (c) OeWF<\/p><\/div>\n

\"Cameras'<\/a>

Cameras’ trajectory and View map (c) OeWF<\/p><\/div>\n

Support Vector Regression<\/b><\/p>\n

The main aspect of the project is traversability estimation rather than terrain classification. So, it is inevitable to use a regression model rather than classifying the terrains based on their scores of traversability. It is really important to have a good amount of training data which would play a major role in training a good regression model. For now, we are using some fake data that we have generated by taking some pictures of random terrains and giving score of traversability on the scale of ten. But in the future, for the training data, we can even collect it from a rover and training can be done on the ground station. By doing it this way, we can even annotate the terrain patches automatically from wheel odometry which we can\u2019t do on a spacesuit. The Italian Mars Society is also working to obtain a dataset of annotated terrains which can be used to train the model.<\/p>\n

The features used are simple histograms of gradients extracted from the small patches of terrain. It is much faster even when compared to extraction of 128 dimension SIFT features and much more suitable for terrains. For each patch of the terrain, gradients along the X and Y directions and corresponding orientations are calculated. A histogram of gradient orientations is constructed and given to the SVR for training.<\/p>\n

Conclusion<\/strong><\/p>\n

As of now, we are trying to implement the above-stated methods. But to use it more efficiently in real time in the future, IMU sensors can be used to obtain the trajectory more accurately. As stated above, running the Mapping thread of PTAM on a ground station like on a Rover which is just resting on the Mars or on the nearest space station would be a lot more effective.<\/p>\n

Author: Vishnu Teja Yalakuntla<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

It\u2019s been one month here and I have learnt tons of things working. I wouldn\u2019t say everything is going according to the schedule, but I believe research projects are like this, we hope that something will work and go further […]<\/p>\n","protected":false},"author":2,"featured_media":11223,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[315,45,3],"tags":[270,883,884,882],"class_list":["post-11218","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-guest-blogs","category-oewf-internships","category-oewf-news","tag-intern","tag-support-vector-regression","tag-terrain-estimation","tag-visual-odometry"],"yoast_head":"\nInternship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Internship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)\" \/>\n<meta property=\"og:description\" content=\"It\u2019s been one month here and I have learnt tons of things working. I wouldn\u2019t say everything is going according to the schedule, but I believe research projects are like this, we hope that something will work and go further […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/\" \/>\n<meta property=\"og:site_name\" content=\"Austrian Space Forum (OeWF)\" \/>\n<meta property=\"article:published_time\" content=\"2016-06-21T11:50:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-06-22T08:17:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1279\" \/>\n\t<meta property=\"og:image:height\" content=\"748\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"OeWF\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@oewf\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"OeWF\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/\"},\"author\":{\"name\":\"OeWF\",\"@id\":\"https:\\\/\\\/oewf.org\\\/#\\\/schema\\\/person\\\/d33b1cc5e9a431b7ad8c32cefeaa4aa7\"},\"headline\":\"Internship 2016: Terrain Estimation by a Monocular Camera\",\"datePublished\":\"2016-06-21T11:50:49+00:00\",\"dateModified\":\"2016-06-22T08:17:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/\"},\"wordCount\":877,\"image\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/oewf.org\\\/wp-content\\\/uploads\\\/2016\\\/06\\\/Screenshot-from-2016-06-17-153025.png\",\"keywords\":[\"Intern\",\"support vector regression\",\"terrain estimation\",\"visual odometry\"],\"articleSection\":[\"Guest blogs\",\"Internships at the OeWF\",\"OeWF News\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/\",\"url\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/\",\"name\":\"Internship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/oewf.org\\\/wp-content\\\/uploads\\\/2016\\\/06\\\/Screenshot-from-2016-06-17-153025.png\",\"datePublished\":\"2016-06-21T11:50:49+00:00\",\"dateModified\":\"2016-06-22T08:17:19+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/#\\\/schema\\\/person\\\/d33b1cc5e9a431b7ad8c32cefeaa4aa7\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#primaryimage\",\"url\":\"https:\\\/\\\/oewf.org\\\/wp-content\\\/uploads\\\/2016\\\/06\\\/Screenshot-from-2016-06-17-153025.png\",\"contentUrl\":\"https:\\\/\\\/oewf.org\\\/wp-content\\\/uploads\\\/2016\\\/06\\\/Screenshot-from-2016-06-17-153025.png\",\"width\":1279,\"height\":748,\"caption\":\"Cameras' trajectory and View map (c) OeWF\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/oewf.org\\\/en\\\/2016\\\/06\\\/internship-2016-terrain-estimation-monocular-camera\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\\\/\\\/oewf.org\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Internship 2016: Terrain Estimation by a Monocular Camera\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/oewf.org\\\/#website\",\"url\":\"https:\\\/\\\/oewf.org\\\/\",\"name\":\"Austrian Space Forum (OeWF)\",\"description\":\"\u00d6sterreichisches Weltraum Forum Austrian Space Forum\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/oewf.org\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/oewf.org\\\/#\\\/schema\\\/person\\\/d33b1cc5e9a431b7ad8c32cefeaa4aa7\",\"name\":\"OeWF\",\"sameAs\":[\"https:\\\/\\\/www.oewf.org\",\"https:\\\/\\\/x.com\\\/oewf\"],\"url\":\"https:\\\/\\\/oewf.org\\\/en\\\/author\\\/oewf\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Internship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/","og_locale":"en_US","og_type":"article","og_title":"Internship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)","og_description":"It\u2019s been one month here and I have learnt tons of things working. I wouldn\u2019t say everything is going according to the schedule, but I believe research projects are like this, we hope that something will work and go further […]","og_url":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/","og_site_name":"Austrian Space Forum (OeWF)","article_published_time":"2016-06-21T11:50:49+00:00","article_modified_time":"2016-06-22T08:17:19+00:00","og_image":[{"width":1279,"height":748,"url":"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png","type":"image\/png"}],"author":"OeWF","twitter_card":"summary_large_image","twitter_creator":"@oewf","twitter_misc":{"Written by":"OeWF","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#article","isPartOf":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/"},"author":{"name":"OeWF","@id":"https:\/\/oewf.org\/#\/schema\/person\/d33b1cc5e9a431b7ad8c32cefeaa4aa7"},"headline":"Internship 2016: Terrain Estimation by a Monocular Camera","datePublished":"2016-06-21T11:50:49+00:00","dateModified":"2016-06-22T08:17:19+00:00","mainEntityOfPage":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/"},"wordCount":877,"image":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#primaryimage"},"thumbnailUrl":"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png","keywords":["Intern","support vector regression","terrain estimation","visual odometry"],"articleSection":["Guest blogs","Internships at the OeWF","OeWF News"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/","url":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/","name":"Internship 2016: Terrain Estimation by a Monocular Camera - Austrian Space Forum (OeWF)","isPartOf":{"@id":"https:\/\/oewf.org\/#website"},"primaryImageOfPage":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#primaryimage"},"image":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#primaryimage"},"thumbnailUrl":"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png","datePublished":"2016-06-21T11:50:49+00:00","dateModified":"2016-06-22T08:17:19+00:00","author":{"@id":"https:\/\/oewf.org\/#\/schema\/person\/d33b1cc5e9a431b7ad8c32cefeaa4aa7"},"breadcrumb":{"@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#primaryimage","url":"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png","contentUrl":"https:\/\/oewf.org\/wp-content\/uploads\/2016\/06\/Screenshot-from-2016-06-17-153025.png","width":1279,"height":748,"caption":"Cameras' trajectory and View map (c) OeWF"},{"@type":"BreadcrumbList","@id":"https:\/\/oewf.org\/en\/2016\/06\/internship-2016-terrain-estimation-monocular-camera\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/oewf.org\/en\/"},{"@type":"ListItem","position":2,"name":"Internship 2016: Terrain Estimation by a Monocular Camera"}]},{"@type":"WebSite","@id":"https:\/\/oewf.org\/#website","url":"https:\/\/oewf.org\/","name":"Austrian Space Forum (OeWF)","description":"\u00d6sterreichisches Weltraum Forum Austrian Space Forum","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/oewf.org\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/oewf.org\/#\/schema\/person\/d33b1cc5e9a431b7ad8c32cefeaa4aa7","name":"OeWF","sameAs":["https:\/\/www.oewf.org","https:\/\/x.com\/oewf"],"url":"https:\/\/oewf.org\/en\/author\/oewf\/"}]}},"_links":{"self":[{"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/posts\/11218","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/comments?post=11218"}],"version-history":[{"count":0,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/posts\/11218\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/media\/11223"}],"wp:attachment":[{"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/media?parent=11218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/categories?post=11218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oewf.org\/en\/wp-json\/wp\/v2\/tags?post=11218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}