\n
<\/p>\n\n\n\n
Rendering of a chair instance from ShapeNet using the voxel-based representation from Plenoxel.<\/p>\n<\/div><\/div>\n\n\n\n
We are generating the grids for all object instances for categories in ShapeNet following which we plan to train our AutSDF on such 3D representations<\/p>\n\n\n\n
Shape Primitives<\/h1>\n\n\n\n Using an autoregressive transformer over a set of latent symbols can be thought of as learning a language of shapes<\/em>. Inspired by this thought, we plan to use shape-primitives (cylinder, cone, plane, etc.) as building blocks of modern shapes. The aim is to learn an autoregressive model (similar to the shape prior in AutoSDF) over such primitives. We first use the 3D CAD models from the ABC dataset and aim to extract surfaces of primitive categories. <\/p>\n\n\n\nRow I: u-v points plotted based on the primitives; Row II: Plotting surfaces (with limits extracted to reduce parameters) <\/figcaption><\/figure>\n\n\n\nWe are still finding a way to extract a tree-like generative sequence to create shapes from primitives. Once extracted, we can restructure the data and train a shape prior based on such primitives.<\/p>\n","protected":false},"excerpt":{"rendered":"
Our proposed approach AutoSDF can faithfully capture the geometry of shapes from the ShapeNet dataset. However, it is limited by the need for volumetric 3D shape representations for training or fine-tuning. We propose to learn the appearance of objects (color) together with geometry (vanilla AutoSDF) to enable extension to novel categories where volumetric 3D is … <\/p>\n
Continue reading “Novel Categories”<\/span><\/a><\/p>\n","protected":false},"author":100,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"class_list":["post-150","post","type-post","status-publish","format-standard","hentry","category-extension"],"yoast_head":"\nNovel Categories - Autoregressive Conditional Generation using Transformers<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Novel Categories - Autoregressive Conditional Generation using Transformers\" \/>\n<meta property=\"og:description\" content=\"Our proposed approach AutoSDF can faithfully capture the geometry of shapes from the ShapeNet dataset. However, it is limited by the need for volumetric 3D shape representations for training or fine-tuning. We propose to learn the appearance of objects (color) together with geometry (vanilla AutoSDF) to enable extension to novel categories where volumetric 3D is … Continue reading "Novel Categories"\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/\" \/>\n<meta property=\"og:site_name\" content=\"Autoregressive Conditional Generation using Transformers\" \/>\n<meta property=\"article:published_time\" content=\"2022-04-29T19:13:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-04-29T19:13:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives-1024x436.png\" \/>\n<meta name=\"author\" content=\"paritosm\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"paritosm\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/\"},\"author\":{\"name\":\"paritosm\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/#\\\/schema\\\/person\\\/0a6afd0e4ed412755a2749c183fbc369\"},\"headline\":\"Novel Categories\",\"datePublished\":\"2022-04-29T19:13:19+00:00\",\"dateModified\":\"2022-04-29T19:13:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/\"},\"wordCount\":267,\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/wp-content\\\/uploads\\\/sites\\\/47\\\/2022\\\/04\\\/initial-primitives-1024x436.png\",\"articleSection\":[\"AutoSDF extension\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/\",\"name\":\"Novel Categories - Autoregressive Conditional Generation using Transformers\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/wp-content\\\/uploads\\\/sites\\\/47\\\/2022\\\/04\\\/initial-primitives-1024x436.png\",\"datePublished\":\"2022-04-29T19:13:19+00:00\",\"dateModified\":\"2022-04-29T19:13:21+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/#\\\/schema\\\/person\\\/0a6afd0e4ed412755a2749c183fbc369\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#primaryimage\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/wp-content\\\/uploads\\\/sites\\\/47\\\/2022\\\/04\\\/initial-primitives.png\",\"contentUrl\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/wp-content\\\/uploads\\\/sites\\\/47\\\/2022\\\/04\\\/initial-primitives.png\",\"width\":2324,\"height\":990},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/2022\\\/04\\\/29\\\/novel-categories\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Novel Categories\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/#website\",\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/\",\"name\":\"Autoregressive Conditional Generation using Transformers\",\"description\":\"Students: Yen-Chi Cheng, Paritosh Mittal | Advisors: Maneesh Singh (Verisk), Shubham Tulsiani (CMU) | Sponsor: Verisk\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/#\\\/schema\\\/person\\\/0a6afd0e4ed412755a2749c183fbc369\",\"name\":\"paritosm\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g\",\"caption\":\"paritosm\"},\"url\":\"https:\\\/\\\/mscvprojects.ri.cmu.edu\\\/2021teamb\\\/author\\\/paritosm\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Novel Categories - Autoregressive Conditional Generation using Transformers","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/","og_locale":"en_US","og_type":"article","og_title":"Novel Categories - Autoregressive Conditional Generation using Transformers","og_description":"Our proposed approach AutoSDF can faithfully capture the geometry of shapes from the ShapeNet dataset. However, it is limited by the need for volumetric 3D shape representations for training or fine-tuning. We propose to learn the appearance of objects (color) together with geometry (vanilla AutoSDF) to enable extension to novel categories where volumetric 3D is … Continue reading \"Novel Categories\"","og_url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/","og_site_name":"Autoregressive Conditional Generation using Transformers","article_published_time":"2022-04-29T19:13:19+00:00","article_modified_time":"2022-04-29T19:13:21+00:00","og_image":[{"url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives-1024x436.png","type":"","width":"","height":""}],"author":"paritosm","twitter_card":"summary_large_image","twitter_misc":{"Written by":"paritosm","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#article","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/"},"author":{"name":"paritosm","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/#\/schema\/person\/0a6afd0e4ed412755a2749c183fbc369"},"headline":"Novel Categories","datePublished":"2022-04-29T19:13:19+00:00","dateModified":"2022-04-29T19:13:21+00:00","mainEntityOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/"},"wordCount":267,"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives-1024x436.png","articleSection":["AutoSDF extension"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/","name":"Novel Categories - Autoregressive Conditional Generation using Transformers","isPartOf":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/#website"},"primaryImageOfPage":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#primaryimage"},"image":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#primaryimage"},"thumbnailUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives-1024x436.png","datePublished":"2022-04-29T19:13:19+00:00","dateModified":"2022-04-29T19:13:21+00:00","author":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/#\/schema\/person\/0a6afd0e4ed412755a2749c183fbc369"},"breadcrumb":{"@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#primaryimage","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives.png","contentUrl":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/47\/2022\/04\/initial-primitives.png","width":2324,"height":990},{"@type":"BreadcrumbList","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/2022\/04\/29\/novel-categories\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/"},{"@type":"ListItem","position":2,"name":"Novel Categories"}]},{"@type":"WebSite","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/#website","url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/","name":"Autoregressive Conditional Generation using Transformers","description":"Students: Yen-Chi Cheng, Paritosh Mittal | Advisors: Maneesh Singh (Verisk), Shubham Tulsiani (CMU) | Sponsor: Verisk","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/#\/schema\/person\/0a6afd0e4ed412755a2749c183fbc369","name":"paritosm","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e788c1e87110c6034ea67b5363857f33e54190080bcb97049b1dc788fa1bd9a9?s=96&d=mm&r=g","caption":"paritosm"},"url":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/author\/paritosm\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/posts\/150","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/users\/100"}],"replies":[{"embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/comments?post=150"}],"version-history":[{"count":1,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/posts\/150\/revisions"}],"predecessor-version":[{"id":153,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/posts\/150\/revisions\/153"}],"wp:attachment":[{"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/media?parent=150"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/categories?post=150"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mscvprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/tags?post=150"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}