Increase the size of galactic and star bases

Discussion forum for Celestia developers; topics may only be started by members of the developers group, but anyone can post replies.
Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Increase the size of galactic and star bases

Post #1by Art Blos » 18.03.2019, 10:59

There are unpleasant considerations that 10 thousand galaxies and 2 million stars by modern standards are too few. :sad:
Many simulators on these indicators left us far behind.

I am sure there will be people who can increase the sizes of bases several times. The task is not to add all existing objects (even a billion from Gaia is too much). But at least millions and tens of millions, theoretically, can be generated. Anyone take up this?
Founder and head of the project "Celestia Origin"

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #2by Lafuente_Astronomy » 18.03.2019, 12:44

Well, for the Galaxies, the closest we can get are some of Selden's addons in the "Data Plots" section in the Celestia Motherlode: http://celestiamotherlode.net/catalog/dataplots.php

As for Stars, it's just a matter of adding STC or SSC files. I actually wanted to try and make some based on the Gaia Catalog but alas, much of the files there are beyond my understanding. If anyone from here can manage to make the Gaia Archive files understandble enough, perhaps I can try to make an STC file

And while a billion is too much (Around 150+ GB I believe), it might be worth asking the developers of Celestia 1.7.0 to improve the graphics of Celestia to handle more than billions worth of Celestial objects. Because we're not just going to count stars and galaxies, we're going to count everything else Science has discovered eversince we first started looking at the sky.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #3by Art Blos » 18.03.2019, 12:55

Lafuente_Astronomy wrote:http://celestiamotherlode.net/catalog/dataplots.php
This is not what is needed. Speech about galaxies.dsc.

Lafuente_Astronomy wrote:And while a billion is too much (Around 150+ GB I believe)
It was calculated that about 20 GB (which is still very heavy for Celestia). Optimally for stars.dat would be about 100 megabytes.
Founder and head of the project "Celestia Origin"

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #4by Lafuente_Astronomy » 18.03.2019, 13:10

Art Blos wrote:This is not what is needed. Speech about galaxies.dsc.

True. I'm just stating that that's the closest thing we have to a numerous "galaxies" file. Until one works on making them more than just data plots, it will remain that way.

Art Blos wrote:It was calculated that about 20 GB (which is still very heavy for Celestia). Optimally for stars.dat would be about 100 megabytes.

Which is of course, a problem that we have to overcome if we are to include millions of stars in the catalogs. Celestia is an outdated but functional program, and I'm afraid to say that its limits make it an abacus compared to the Supercomputers like Gaia Sky, Starry Night, SpaceEngine, IllustrisTNG, etc

Still, compared to the others, I enjoy Celestia the most. That's why I'm rooting for you guys to make it better.

Added after 10 minutes 57 seconds:
Art Blos, is it ok if I send this link to the "Time to Restart Development" post? I do think your question is necessary for the development of Celestia 1.7.0 after all.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #5by Art Blos » 18.03.2019, 13:24

Lafuente_Astronomy wrote:Art Blos, is it ok if I send this link to the "Time to Restart Development" post? I do think your question is necessary for the development of Celestia 1.7.0 after all.
It should be noticed anyway.
Another thing is that 1.7.0 is still unknown when it comes out. We must continue to improve 1.6.1
Founder and head of the project "Celestia Origin"

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #6by Lafuente_Astronomy » 18.03.2019, 13:30

Art Blos wrote:Another thing is that 1.7.0 is still unknown when it comes out. We must continue to improve 1.6.1

Well, Development has been slow as a result of the number of programmers working on it. I'm just stating that your considerations for an improved stars and galaxies base files could be considered as an objective for the development of 1.7.0. So that once it's released, there would be no need to make an addon updating most of those files with information 10 years and older, rather, one can make addons to supplement more recent discoveries and catalogs into it instead.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #7by Lafuente_Astronomy » 20.03.2019, 22:05

Well, there is also a new problem, as Joey P pointed out here: viewtopic.php?f=2&t=16439&p=142659#p142659

Aside from increasing our stars and galaxies database, we must also ensure that they are scientifically accurate, i.e based on actual science information on those objects. A lot of objects in Celestia have their sizes wrong, some even impossibly wrong. One of those examples is Mu Cephi, as seen here:
Impossibly huge size.jpg

As you can see, its Radius size is at an impossible 3900 Rsun! If I remember, the actual largest discovered star, UY Scuti, only has a radi of 1800+Rsun. So, this one had to be the wrong size for the star.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

onetwothree
Developer
Posts: 324
Joined: 22.09.2018
With us: 1 year 1 month

Post #8by onetwothree » 22.03.2019, 08:56

You don't need to be a programmer to fix such issues. We are waiting for contributions.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #9by Art Blos » 22.03.2019, 09:02

Hey, I remember exactly here published a two million stars, based on data from Gaia. The question is, how was it created and edited? And is it possible to make the same variant with a large number of objects?

Added after 1 minute 57 seconds:
It is a pity that no one thought of putting all the necessary tools in a compiled form so that people far from programming could use them. This applies to generators of binary star base, galaxies, globular clusters, and so on.
Founder and head of the project "Celestia Origin"

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #10by Lafuente_Astronomy » 22.03.2019, 09:56

Art Blos wrote:It is a pity that no one thought of putting all the necessary tools in a compiled form so that people far from programming could use them. This applies to generators of binary star base, galaxies, globular clusters, and so on.

I can relate. I though I can improve the stars.dat file because it would look like an STC file but with a complete catalog. But when I loaded it, it's mostly written in characters that do not exist in my keyboard.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

onetwothree
Developer
Posts: 324
Joined: 22.09.2018
With us: 1 year 1 month

Post #11by onetwothree » 22.03.2019, 12:05

Binary stars.dat is built from stars.txt using src/tools/stardb/buildstardb.pl (requires Perl) or makestardb provided in celestia-tools archive.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #12by Art Blos » 22.03.2019, 17:48

onetwothree wrote:Binary stars.dat is built from stars.txt using src/tools/stardb/buildstardb.pl (requires Perl) or makestardb provided in celestia-tools archive.
All this is known. It did not give concrete results. Greg tried to compile, but failed.
Founder and head of the project "Celestia Origin"

onetwothree
Developer
Posts: 324
Joined: 22.09.2018
With us: 1 year 1 month

Post #13by onetwothree » 22.03.2019, 19:18

Art Blos wrote:Greg tried to compile, but failed.

Compiled makestardb, makexindex and startextdump are distributed in a compiled form from our bintray repository https://dl.bintray.com/celestia/celestia-builds/:

celestia-tools-win32_1.7.0~git20190120+706c99d.7z
celestia-tools-win64_1.7.0~git20190120+706c99d.7z

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #14by Art Blos » 22.03.2019, 19:42

onetwothree wrote:Compiled makestardb, makexindex and startextdump are distributed in a compiled form from our bintray repository
Hm... Thanks, I will try to test them (at least some).
Founder and head of the project "Celestia Origin"

Avatar
LukeCEL
Posts: 197
Joined: 26.09.2017
With us: 2 years 1 month

Post #15by LukeCEL » 24.03.2019, 22:21

Here's a stars.dat that contains stars from the Hipparcos and Tycho-2 catalogues, with distances from Gaia DR2 or Hipparcos parallaxes. All in all, it has 2,448,220 stars. This is close to the maximum amount of real stars that can be added to Celestia, since the stars.dat format only works for the TYC and HIP catalogues, which only have about this much stars.

To make this, I simply tweaked my buildstardb.pl file enough that it could read data from a different source. (Actually, my buildstardb.pl is an already modified version of what's in the source code.) It reads data from the usual sources, but also CDS' XMatch tool, which allows for relatively quick cross-matching between two tables. So I did "Tycho-2" for the first table, and "SIMBAD" for the second table. The resulting table is about 1 GB in size.

Since XMatch doesn't allow me to include the parallax error, many of the Tycho stars included may have less-than-ideal parallaxes. A few may even be from other galaxies.

Any feedback is welcome, though I should note that it took my computer over 20 minutes to build these two files, so it might take a long time for me to implement suggestions.
Attachments
stars.dat.zip
(39.62 MiB) Downloaded 32 times
stars.txt.zip
(62.99 MiB) Downloaded 32 times

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #16by Lafuente_Astronomy » 25.03.2019, 05:00

LukeCEL wrote:Any feedback is welcome, though I should note that it took my computer over 20 minutes to build these two files, so it might take a long time for me to implement suggestions.

Well, is it possible to add stars from the Gaia Archives even though they are not part of either Tycho or Simbad? And not the whole archives themselves, at least just enough stars to populate the Milky Way Galaxy without affecting the Optimal speed of Celestia
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #17by Art Blos » 25.03.2019, 08:06

LukeCEL wrote:This is close to the maximum amount of real stars that can be added to Celestia, since the stars.dat format only works for the TYC and HIP catalogues, which only have about this much stars.
This is very important information. I did not know that the format has limitations. I wanted to make all available stars binary (i.e. non-textual), but alas. :sad:

LukeCEL wrote:Any feedback is welcome
You have reached my main goal - got rid of phantoms TYC XXXX-XXXXX-0. You are a miracle! :clap: :clap: :clap:
And what about the galaxies? :smile:
Founder and head of the project "Celestia Origin"

Avatar
LukeCEL
Posts: 197
Joined: 26.09.2017
With us: 2 years 1 month

Post #18by LukeCEL » 25.03.2019, 22:01

Lafuente_Astronomy wrote:Well, is it possible to add stars from the Gaia Archives even though they are not part of either Tycho or Simbad?

Art Blos mentioned this, but 2.5 million is the number of real stars you can add using stars.dat. I guess you can add as many stars as you want using STC files, but STC files are much slower to load compared to DAT. You definitely don't want to be adding millions of stars using STC.

Art Blos wrote:And what about the galaxies?
This is on my to-do list, but I need to find a sufficient data source for this.

Avatar
Lafuente_Astronomy
Moderator
Posts: 436
Joined: 04.08.2018
Age: 21
With us: 1 year 3 months
Location: Cebu City, Cebu Province, Philippines
Contact:

Post #19by Lafuente_Astronomy » 25.03.2019, 22:15

LukeCEL wrote:Art Blos mentioned this, but 2.5 million is the number of real stars you can add using stars.dat.

Oh. Well, hopefully there's a way to increase that in the star.dat file for Celestia 1.7.0, as a way of modernizing it and making it powerful enough to keep up with the newer and more advance planetarium software.
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Avatar
Topic author
Art Blos M
Moderator
Posts: 639
Joined: 31.08.2017
Age: 27
With us: 2 years 2 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #20by Art Blos » 26.03.2019, 05:41

LukeCEL wrote:This is on my to-do list, but I need to find a sufficient data source for this.
The source from which the old base was made is no longer suitable?
Founder and head of the project "Celestia Origin"


Return to “Ideas & News”

Who is online