Creating on spot and online digital twins with ARE4
Step-by-step guide in your journey to augment your favorite places and turn them into digital twins = real-world social network where your place becomes the canvas for on spot and online UGC
At Neogoma, our vision is to merge the best of both online and offline worlds. Merging means we shouldn’t have to choose between the infinite possibilities offered by digital platforms and the beauty of real-world interactions. We want to have both of them. That’s why we created ARE4, a real-world social network where your favorite places are becoming a canvas for 3D UGC — user-generated content created on spot AND online and vice versa, automatically synchronized.
Example
If you happen, for example during your next holidays, to pass by the amazingly cool climbing wall Palpa Roca in Spain, you’ll be able to enjoy the full experience thanks to world-scale, ultra-accurate and persistent Augmented reality (the 3D content is precisely pinned to the venue).
However, since we assume most of the readers are scattered all around the world, you can only join the online experience by clicking here and here.
The content (3D painting) constantly evolves. It’s part of the experience for both place owners and their visitors. We don’t want those visitors to stay spectators, they become actors of their online and offline experiences.
We ask you to respect the work of digital artists who worked on this project. You can obviously leave your mark — that’s the entire point of the web editor-, but the place owner keeps the right to filter your creations.
Creating your own digital twin is as simple as creating a post on social media. Takes a few minutes only.
Basic requirements
To create a digital twin of your favorite place, there are basically 3 things you need:
- A cool place you want to map. Start with small places with lots of colors before moving to large spaces with white & reflective walls (more complicated to obtain good results). You don’t have to own a place to map it (but you might want to explain your intentions to the owner)
- An account which can be created within the app (App store / Play store) or on the web dashboard. You’ll need a valid email address to verify your account.
- A decent cell phone with a good camera resolution. We recommend most iPhones and flagship phones from Samsung, Pixel, OnePlus or Asus. Other Android phones (e.g. Huawei, Nokia, Sony, LG, HTC, Motorola, Lenovo, Xiaomi, Realme, Oppo, Sharp, Vivo, Zebra) are usually working but it’s advised to check if they support Google’s latest ARcore version.
ARE4 mobile app
Assuming you’ve downloaded the app, you can start exploring other users’ creations that might inspire you.
As a digital twin creator, you’re about to become a mapper! Click on the mapping icon on the bottom left and name your first place. Note that you don’t need an active internet connection, so you can map in airplane mode.
You’ll need to allow camera access and agree on the Augmented Reality warning. Those are mandatory by app stores. Once you enter the camera mode and hit the start button, mapping pictures will be automatically taken based on your movements — a combination of rotation and distance. The picture counter on the top, as well as the progress bar on the bottom, tell you how many pictures you’ve captured. 100 pictures should be good enough to start with.
Now before jumping into 3D painting, there are a few recommendations we’d like to insist on, as the quality of relocation and reconstruction depends on the mapping phase.
- Avoid blur! Move slowly, don’t shake your device and make sure your camera autofocuses
- Avoid sharp turns! You will need to turn, but do it slowly
- Avoid pivotal viewpoints (staying without moving and simply rotating)
- Avoid reflective surfaces (glass, floors with light reflections) and white empty walls
- Outside in: best mapping strategy to map most of the places as explained in this video.
Congrats, you’ve just mapped your first place! Before processing it, you can already create some content on spot: welcome to painting in 3D.
You can select different brush types, sizes and colors. Simply hit the undo button if you don’t like a paint stroke. The only limit is your creativity!
You’re almost good to go! Click on the “save” button and confirm by hitting the “finish” button and you’ll be redirected to the upload screen. Since the entire mapping process doesn’t require an internet connection, you might as well save your place locally on your phone and upload it later. If you can’t wait, select a thumbnail picture and decide on privacy before uploading your map to ARE4’s servers.
If you decided to wait for a WIFI connection, you can go to your user center (bottom right button), click “On device”, then click on your map and it will bring you back to that same upload screen as shown above.
Upload speed depends on your internet connection. DO NOT CLOSE THE APP WHILE UPLOADING.
The next step is to wait… But not for long! You’ll get notified when your place is ready. Basically ARE4 will inform you when you and your friend can explore content on spot (thanks to ultra-accurate relocation) and collaborate online (using the 3D reconstruction as the canvas). Usually the relocation notification comes first as 3D reconstruction requires more computing time.
The link opens your place directly in ARE4 app if one has the app installed, if not it opens the web viewer (see next part).
Assuming you have the ARE4 app, you can also go to map / feed discover screens if your map is public. If it’s private, go to your user center to open it. Note that the AR button will be deactivated until you get notified. Then comes the fun! Click on “Yes, let’s explore” to open your camera for ultra-accurate positioning.
Depending on your viewpoint, on your camera resolution or on lighting conditions, you can get 3 relocation statuses for each of your requests: green means excellent accuracy, yellow is still good but could be better, red signifies we either can’t relocate your device or noticed the offset is too big. You can relocate as many times as you want, especially if you know exactly where your content should be.
You’re already familiar with the painting process by now, so bring your friends in for a more collaborative experience on spot!
There is also a painting repositioning mode but it requires more experience as you’ll probably lose your sense of distance. For experts, it’s a great tool to position paintings on inaccessible places like buildings’ roofs.
Last word on the app side, going back to your user center, you can manage your place name, privacy and location (address).
This should get your started! If you still have doubts, we’ve prepared a few tutorial, check that below 👇
Unfortunately, we know not everyone can go on spot to participate in the experience. But we’ve designed ARE4 platform with that in mind, so just share the link and let them join the online experience as explained below!
ARE4 web viewer and editor
For this part of the guide, we’ll be using the same place as the above “favorite place”, so feel free to open it on a browser. We recommend using a computer rather than a mobile phone browser, even if it works too.
Public version: place explorer
Anyone opening your place on a browser will land “within” your place. Your visitors don’t need to be registered to enjoy the experience, we wanted to make it as open as possible.
Now since this might be the first step into spatial computing, visitors should click on the panel button (right middle of the screen) and help button to learn of to navigate within the scene. Navigation mode is the default mode, which allows visitors to rotate the place, zoom in and out. For a greater sense of immersion, QWEASD keys are moving the camera — like in video games.
To start painting, you’ll need to click on the brush icon from that panel. When selected, a mode, style or color is always brighter than the other buttons. As in the app, you can choose different brushes, colors and sizes. To understand what is the canvas distance, you’ll need to imagine the distance from your screen to your mouse cursor as if your mouse was “inside” your place. The 3 buttons on the bottom are pretty straightforward: Undo a stroke, reset the default view, save your paintings.
Warning: once the paintings are saved, they can no longer be manipulated.
We’re not gonna lie: there is a learning curve to painting in 3D. After all you have to take a new dimension into account which could sometimes be counter-intuitive. But with a bit of practice, everything gets easy! Or maybe you’re already the Picasso of 3D?
For straight or streamer style lines, just maintain the painting click while navigating with the QWEASD keys:
The last mode is object manipulation. *BEFORE SAVING* you can select any of your painting strokes and reposition it, rescale it, re-orient it. This is almost professional level but hey, if you made it here, you’re on the verge of mastering 3D content! The gizmo allows you to do pretty much everything needed… but remember you are manipulating 3D objects from a 2D screen, so it can become tricky! Everything depends on which perspective you’re looking at it 😅.
Side note, at Neogoma, we’re on a mission to democratize spatial computing and 3D, so we’re not forcing visitors to register to paint the world. Everything above can be done by anyone, anywhere!
For place owners: dashboard and editor
Registering or signing in (if you already created an account on the app, use the same email address and password) allows you access to additional features but most important allows you to manage content.
Once logged in, you’ll see the list of all your places sorted by default by creation time. You can sort them out differently by clicking on a place name, relocation and reconstruction status, views or relocations. It’s also easy to copy and paste the public link of your place.
As the place owner you’ll need to manage the UGC created in your place — the same way you have to delete hate messages on social media. You will get notified by ARE4 every time a visitor saves new content.
As you may have noticed, there are 2 links on your dashboard: https://dashboard.are4.world/editor/6433c48ef0bf7 vs https://place.are4.world/6433c48ef0bf7 They are almost similar and yet there is a major difference: the first link under the column “manage content” has delete button! Only you, as the place owner, can manage content. Apart from deleting, you can also manipulate content (rescale, reposition, reorient, etc).
That covers the web editor part. As with the app, go to our Youtube channel for more tutorials.
If you have any questions, issues or suggestions, the best way would be to share them with us on Discord! That’s also where you can show your places and ask designers and artists to jump in your digital twins.
That’s it for now, happy world augmentation!
