{"id":168,"date":"2024-05-21T09:43:06","date_gmt":"2024-05-21T09:43:06","guid":{"rendered":"https:\/\/wsw-int.de\/?p=168"},"modified":"2024-10-25T15:23:12","modified_gmt":"2024-10-25T15:23:12","slug":"%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8","status":"publish","type":"post","link":"https:\/\/multai.eu\/de\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/","title":{"rendered":"Chameleon, ein gemischt-modales Early-Fusion-Grundlagenmodell"},"content":{"rendered":"<p>In a new <a href=\"https:\/\/arxiv.org\/pdf\/2405.09818\">paper<\/a>, Meta announces \ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27, a \ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\ud835\udc1d\ud835\udc1a\ud835\udc25 \ud835\udc1e\ud835\udc1a\ud835\udc2b\ud835\udc25\ud835\udc32-\ud835\udc1f\ud835\udc2e\ud835\udc2c\ud835\udc22\ud835\udc28\ud835\udc27 foundation model. Contrary to earlier multimodal models, which model the different modalities (text, image, audio, etc.) separately, mixed-modal early-fusion foundation models like Chameleon are end-to-end models. They ingest all modalities from the start and project them into one representational space. That permits integrating information across modalities and generating multimodal documents. Indeed, the paper contains some nice examples of \ud835\udc22\ud835\udc27\ud835\udc2d\ud835\udc1e\ud835\udc2b\ud835\udc25\ud835\udc1e\ud835\udc1a\ud835\udc2f\ud835\udc1e\ud835\udc1d \ud835\udc22\ud835\udc26\ud835\udc1a\ud835\udc20\ud835\udc1e \ud835\udc1a\ud835\udc27\ud835\udc1d \ud835\udc2d\ud835\udc1e\ud835\udc31\ud835\udc2d generation (see below), which seems to be Chameleon\u2019s forte.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"970\" src=\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\" alt=\"\" class=\"wp-image-169\" srcset=\"https:\/\/multai.eu\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png 1024w, https:\/\/multai.eu\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-300x284.png 300w, https:\/\/multai.eu\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-768x728.png 768w, https:\/\/multai.eu\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332.png 1184w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Interleaved query and response (from the paper).<\/figcaption><\/figure>\n\n\n\n<p>Despite Meta\u2019s different practices in the social media department, the company is remarkably transparent in its GenAI business. The paper contains many interesting insights, and the Chameleon model will hopefully become open source as its predecessors already are.<\/p>\n\n\n\n<p>The paper describes the used datasets (not including Meta user data) and gives detailed insights into techniques for stable and scalable model training. Interesting too is the section about inference that identifies \ud835\udc2d\ud835\udc21\ud835\udc2b\ud835\udc1e\ud835\udc1e \ud835\udc2c\ud835\udc29\ud835\udc1e\ud835\udc1c\ud835\udc22\ud835\udc1f\ud835\udc22\ud835\udc1c \ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\ud835\udc1d\ud835\udc1a\ud835\udc25 \ud835\udc1c\ud835\udc21\ud835\udc1a\ud835\udc25\ud835\udc25\ud835\udc1e\ud835\udc27\ud835\udc20\ud835\udc1e\ud835\udc2c: generated tokens must be copied from the GPU to the CPU to inspect their nature (i.e., text or image) to send them to the correct decoder, tokens that do not belong to a particular modality need to be masked, and finally, text\u2019s variable length versus images\u2019 fixed-size blocks of tokens need to be seamlessly integrated.<\/p>\n\n\n\n<p>As for \ud835\udc1e\ud835\udc2f\ud835\udc1a\ud835\udc25\ud835\udc2e\ud835\udc1a\ud835\udc2d\ud835\udc22\ud835\udc28\ud835\udc27, Chameleon excels at interleaved image and text generation, while remaining very competitive in text-only and image-only tasks. Missing is a comparison against GPT-4o (to be fair, launched only three days before this paper was published, indicative of the speed of innovation). Unfortunately, not much is known about GPT-4o\u2019s architecture. Likely, GPT-4o is much larger than the 34 billion parameter Chameleon (which also exists in a 7B version) and trained on more data. If Chameleon holds up to GPT-4o, then it might lead us towards a future of smaller models, which is desirable in many ways. Note, however, that GPT-4o has audio capabilities which are currently absent in Chameleon.<\/p>\n\n\n\n<p>Next to benchmarking (for the text-only and image-to-text tasks) for evaluation purposes, the paper contains a large section about \ud835\udc21\ud835\udc2e\ud835\udc26\ud835\udc1a\ud835\udc27 \ud835\udc1e\ud835\udc2f\ud835\udc1a\ud835\udc25\ud835\udc2e\ud835\udc1a\ud835\udc2d\ud835\udc22\ud835\udc28\ud835\udc27, which serves as a great introduction to the topic for the uninitiated reader.<\/p>\n\n\n\n<p><strong><a href=\"https:\/\/multai.eu\/de\/\">MultAI.eu <\/a><\/strong>&#8230;<\/p>","protected":false},"excerpt":{"rendered":"<p>In a new paper, Meta announces \ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27, a \ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\ud835\udc1d\ud835\udc1a\ud835\udc25 \ud835\udc1e\ud835\udc1a\ud835\udc2b\ud835\udc25\ud835\udc32-\ud835\udc1f\ud835\udc2e\ud835\udc2c\ud835\udc22\ud835\udc28\ud835\udc27 foundation model. Contrary to earlier multimodal models, which model the different modalities (text, image, audio, etc.) separately, mixed-modal early-fusion foundation models like Chameleon are end-to-end models. They ingest all modalities from the start and project them into one representational space. That permits integrating information across [&hellip;]<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[19,50,10,49,48],"class_list":["post-168","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-genai","tag-interleaved","tag-llm","tag-mixed-modal","tag-multimodal"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Chameleon, a mixed-modal early-fusion foundation model - MultAI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/multai.eu\/de\/\ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27-a-\ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Chameleon, a mixed-modal early-fusion foundation model - MultAI\" \/>\n<meta property=\"og:description\" content=\"In a new paper, Meta announces \ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27, a \ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\ud835\udc1d\ud835\udc1a\ud835\udc25 \ud835\udc1e\ud835\udc1a\ud835\udc2b\ud835\udc25\ud835\udc32-\ud835\udc1f\ud835\udc2e\ud835\udc2c\ud835\udc22\ud835\udc28\ud835\udc27 foundation model. Contrary to earlier multimodal models, which model the different modalities (text, image, audio, etc.) separately, mixed-modal early-fusion foundation models like Chameleon are end-to-end models. They ingest all modalities from the start and project them into one representational space. That permits integrating information across [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/multai.eu\/de\/\ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27-a-\ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\/\" \/>\n<meta property=\"og:site_name\" content=\"MultAI\" \/>\n<meta property=\"article:published_time\" content=\"2024-05-21T09:43:06+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-25T15:23:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\" \/>\n<meta name=\"author\" content=\"hans\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"hans\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"2\u00a0Minuten\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/\"},\"author\":{\"name\":\"hans\",\"@id\":\"https:\/\/multai.eu\/#\/schema\/person\/06def8c374b5d6724bec911e9880c292\"},\"headline\":\"Chameleon, a mixed-modal early-fusion foundation model\",\"datePublished\":\"2024-05-21T09:43:06+00:00\",\"dateModified\":\"2024-10-25T15:23:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/\"},\"wordCount\":368,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/multai.eu\/#organization\"},\"image\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\",\"keywords\":[\"GenAI\",\"interleaved\",\"LLM\",\"mixed-modal\",\"multimodal\"],\"articleSection\":[\"Uncategorized\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/\",\"url\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/\",\"name\":\"Chameleon, a mixed-modal early-fusion foundation model - MultAI\",\"isPartOf\":{\"@id\":\"https:\/\/multai.eu\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\",\"datePublished\":\"2024-05-21T09:43:06+00:00\",\"dateModified\":\"2024-10-25T15:23:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage\",\"url\":\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\",\"contentUrl\":\"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/multai.eu\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Chameleon, a mixed-modal early-fusion foundation model\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/multai.eu\/#website\",\"url\":\"https:\/\/multai.eu\/\",\"name\":\"WSW\",\"description\":\"Generative AI for your business\",\"publisher\":{\"@id\":\"https:\/\/multai.eu\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/multai.eu\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/multai.eu\/#organization\",\"name\":\"WSW\",\"alternateName\":\"MultAI\",\"url\":\"https:\/\/multai.eu\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/multai.eu\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/multai.eu\/wp-content\/uploads\/2024\/10\/Logo.png\",\"contentUrl\":\"https:\/\/multai.eu\/wp-content\/uploads\/2024\/10\/Logo.png\",\"width\":225,\"height\":244,\"caption\":\"WSW\"},\"image\":{\"@id\":\"https:\/\/multai.eu\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/multai.eu\/#\/schema\/person\/06def8c374b5d6724bec911e9880c292\",\"name\":\"hans\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/multai.eu\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1409f6643b6f17d5838709af9deca41643884a95390f8a4f8ea478b9187aec41?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1409f6643b6f17d5838709af9deca41643884a95390f8a4f8ea478b9187aec41?s=96&d=mm&r=g\",\"caption\":\"hans\"},\"sameAs\":[\"https:\/\/wsw-int.de\"],\"url\":\"https:\/\/multai.eu\/de\/author\/hans\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Chameleon, a mixed-modal early-fusion foundation model - MultAI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/multai.eu\/de\/\ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27-a-\ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\/","og_locale":"de_DE","og_type":"article","og_title":"Chameleon, a mixed-modal early-fusion foundation model - MultAI","og_description":"In a new paper, Meta announces \ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27, a \ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\ud835\udc1d\ud835\udc1a\ud835\udc25 \ud835\udc1e\ud835\udc1a\ud835\udc2b\ud835\udc25\ud835\udc32-\ud835\udc1f\ud835\udc2e\ud835\udc2c\ud835\udc22\ud835\udc28\ud835\udc27 foundation model. Contrary to earlier multimodal models, which model the different modalities (text, image, audio, etc.) separately, mixed-modal early-fusion foundation models like Chameleon are end-to-end models. They ingest all modalities from the start and project them into one representational space. That permits integrating information across [&hellip;]","og_url":"https:\/\/multai.eu\/de\/\ud835\udc02\ud835\udc21\ud835\udc1a\ud835\udc26\ud835\udc1e\ud835\udc25\ud835\udc1e\ud835\udc28\ud835\udc27-a-\ud835\udc26\ud835\udc22\ud835\udc31\ud835\udc1e\ud835\udc1d-\ud835\udc26\ud835\udc28\/","og_site_name":"MultAI","article_published_time":"2024-05-21T09:43:06+00:00","article_modified_time":"2024-10-25T15:23:12+00:00","og_image":[{"url":"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png"}],"author":"hans","twitter_card":"summary_large_image","twitter_misc":{"Verfasst von":"hans","Gesch\u00e4tzte Lesezeit":"2\u00a0Minuten"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#article","isPartOf":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/"},"author":{"name":"hans","@id":"https:\/\/multai.eu\/#\/schema\/person\/06def8c374b5d6724bec911e9880c292"},"headline":"Chameleon, a mixed-modal early-fusion foundation model","datePublished":"2024-05-21T09:43:06+00:00","dateModified":"2024-10-25T15:23:12+00:00","mainEntityOfPage":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/"},"wordCount":368,"commentCount":0,"publisher":{"@id":"https:\/\/multai.eu\/#organization"},"image":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage"},"thumbnailUrl":"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png","keywords":["GenAI","interleaved","LLM","mixed-modal","multimodal"],"articleSection":["Uncategorized"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/","url":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/","name":"Chameleon, a mixed-modal early-fusion foundation model - MultAI","isPartOf":{"@id":"https:\/\/multai.eu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage"},"image":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage"},"thumbnailUrl":"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png","datePublished":"2024-05-21T09:43:06+00:00","dateModified":"2024-10-25T15:23:12+00:00","breadcrumb":{"@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#primaryimage","url":"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png","contentUrl":"https:\/\/wsw-int.de\/wp-content\/uploads\/2024\/05\/Screenshot-2024-05-21-112332-1024x970.png"},{"@type":"BreadcrumbList","@id":"https:\/\/multai.eu\/%f0%9d%90%82%f0%9d%90%a1%f0%9d%90%9a%f0%9d%90%a6%f0%9d%90%9e%f0%9d%90%a5%f0%9d%90%9e%f0%9d%90%a8%f0%9d%90%a7-a-%f0%9d%90%a6%f0%9d%90%a2%f0%9d%90%b1%f0%9d%90%9e%f0%9d%90%9d-%f0%9d%90%a6%f0%9d%90%a8\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/multai.eu\/"},{"@type":"ListItem","position":2,"name":"Chameleon, a mixed-modal early-fusion foundation model"}]},{"@type":"WebSite","@id":"https:\/\/multai.eu\/#website","url":"https:\/\/multai.eu\/","name":"WSW","description":"Generative AI for your business","publisher":{"@id":"https:\/\/multai.eu\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/multai.eu\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/multai.eu\/#organization","name":"WSW","alternateName":"MultAI","url":"https:\/\/multai.eu\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/multai.eu\/#\/schema\/logo\/image\/","url":"https:\/\/multai.eu\/wp-content\/uploads\/2024\/10\/Logo.png","contentUrl":"https:\/\/multai.eu\/wp-content\/uploads\/2024\/10\/Logo.png","width":225,"height":244,"caption":"WSW"},"image":{"@id":"https:\/\/multai.eu\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/multai.eu\/#\/schema\/person\/06def8c374b5d6724bec911e9880c292","name":"hans","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/multai.eu\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1409f6643b6f17d5838709af9deca41643884a95390f8a4f8ea478b9187aec41?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1409f6643b6f17d5838709af9deca41643884a95390f8a4f8ea478b9187aec41?s=96&d=mm&r=g","caption":"hans"},"sameAs":["https:\/\/wsw-int.de"],"url":"https:\/\/multai.eu\/de\/author\/hans\/"}]}},"_links":{"self":[{"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/posts\/168","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/comments?post=168"}],"version-history":[{"count":3,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/posts\/168\/revisions"}],"predecessor-version":[{"id":1435,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/posts\/168\/revisions\/1435"}],"wp:attachment":[{"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/media?parent=168"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/categories?post=168"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/multai.eu\/de\/wp-json\/wp\/v2\/tags?post=168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}