New Celestia unified runtime database

The place to discuss creating, porting and modifying Celestia's source code.
Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #121by Lafuente_Astronomy » 21.06.2019, 08:57

Hello Celestians! Especially to the developers

I have a question regarding the GitHub: Does clicking the "Clone or Download" function also downloads all the latest improvements to the AstroDB Branch? Just curious
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #122by pirogronian » 21.06.2019, 09:09

@Lafuente_Astronomy
As it make copy of whole repo, including all branches, so I think yes.

Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #123by Lafuente_Astronomy » 21.06.2019, 10:43

pirogronian wrote:@Lafuente_Astronomy
As it make copy of whole repo, including all branches, so I think yes.

Nice! Well, since we're here, I'd like to ask: How's the developments to the AstroDB Branch going? Any new stuff you both implemented and are planning for the AstroDB Branch?
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #124by pirogronian » 21.06.2019, 17:29

Lafuente_Astronomy wrote:How's the developments to the AstroDB Branch going?

Development stuck up since last changelog. Plans are still the same.

Janus
Posts: 400
Joined: 13.08.2016
With us: 3 years

Post #125by Janus » 21.06.2019, 18:05

A note here from the outside.
Significant portions of the celestia code are based on GLFLOAT as an implicit size of 32-bits.
One of the issues moving to 64-bit was how often int was assumed to be 32-bit, and depended on it.

Changing everything to GLDOUBLE {64-bits}, which is needed to actually use the extra precision, is a long and very involved process.
It is also very recursive, redundant, and ... You get the idea.
As you change one layer, and fix the issues with that change.
You discover another layer, either direct, or indirect, that needs to be changed.
Do not be deceived into thinking you can just recast, that covers the problem without fixing it, or anything.
If even one stage uses float, then the end result is a float, not a double.
This means tracing down every single variable, no matter how it is declared.

1.00f makes a float, 1.00d makes a double.
So you can search for all variable declarations that and in f or F or any other way that defaults to float.
Basically what needs to happen is to change ALL! floats to doubles everywhere at once.
That will actually be less work because of the way portions of the code are cross and inter weaved.

If my current job ever finishes, that is on my list.
However, converting microlimit switch based sequence controls to the logic of functions and procedures is long and involved.
Translating between the motor people, the hydraulic people, weights and measures, management, etc, is hard beyond words.
After finishing this job, converting celestia to pure doubles will be a nice break.


Janus.

Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #126by Lafuente_Astronomy » 22.06.2019, 11:37

In relation to the purpose of the AstroDB, would the entire Gaia Database, both DR1 and DR2 be implemented as an index file in Celestia at some point?
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #127by pirogronian » 22.06.2019, 12:16

Lafuente_Astronomy wrote:In relation to the purpose of the AstroDB, would the entire Gaia Database, both DR1 and DR2 be implemented as an index file in Celestia at some point?

I'm focused on C++ code rather, but who knows...

Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #128by Lafuente_Astronomy » 22.06.2019, 13:28

pirogronian wrote:I'm focused on C++ code rather, but who knows...

Alright. I can wait for that. Also, you think you can do what Janus suggested? It may be one of the ways to improve Celestia's performance with regards to the implementation of huge amounts of data that is associated with the AstroDB Branch.

Added after 9 hours 52 minutes:
Also, on a related note, what is the difference between this: https://github.com/CelestiaProject/Celestia and this: https://github.com/CelestiaProject/Celestia/tree/astrodb ? I'm just wondering, aren't all the improvements compiled to a singular master code file that is the CelestiaProject? If they are not, will they at some point, be merged together to form a singular Master CelestiaProject code file?
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

onetwothree
Developer
Posts: 256
Joined: 22.09.2018
With us: 11 months

Post #129by onetwothree » 26.06.2019, 14:46

Janus wrote:Significant portions of the celestia code are based on GLFLOAT as an implicit size of 32-bits.
One of the issues moving to 64-bit was how often int was assumed to be 32-bit, and depended on it.

Changing everything to GLDOUBLE {64-bits}, which is needed to actually use the extra precision, is a long and very involved process.

I doubt this is a good idea. Doubles are slower, use more memory and the only benefit is more correct depth calculation for large red shifts.

Lafuente_Astronomy wrote:Also, on a related note, what is the difference between this: https://github.com/CelestiaProject/Celestia and this: https://github.com/CelestiaProject/Celestia/tree/astrodb ? I'm just wondering, aren't all the improvements compiled to a singular master code file that is the CelestiaProject? If they are not, will they at some point, be merged together to form a singular Master CelestiaProject code file?

When astrodb branch is ready, it will be integrated into the master branch.

Janus
Posts: 400
Joined: 13.08.2016
With us: 3 years

Post #130by Janus » 26.06.2019, 15:26

@onetwothree

Positioning errors of stars in the database are a byproduct of floating point precision.

Single precision floats have six(6) digits of precision you can depend on, seven(7) you can use if you are careful.
This gives a position error on the order one AU per ~15LY.
Thus it forms a layering pattern when you get way out there.
Coincidentally the distance limit of 10MLY is approximately where minimum positioning errors are close to an LY.

Double precision floating point has 15 digits you can depend on, 16 you can use with care.
The positioning error is then one(1) AU at the edge of the observable universe.

Which means using single precision inside the milkyway is probably fine.
Max position error/drift in our galaxy is ~0.1LY, which is fine for displays.
However, DSOs really need the extra precision if you want to avoid layering.
And for my part, I would like to tinker with doubles even inside the milkyway.


Janus.

onetwothree
Developer
Posts: 256
Joined: 22.09.2018
With us: 11 months

Post #131by onetwothree » 27.06.2019, 09:19

I assume that if you are talking about GLfloat/GLdouble you are talking about OpenGL. Single precision is enough for OpenGL.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #132by pirogronian » 09.07.2019, 19:18

Changelog report:
New names infrastructure seems to be succesfuly implemented for all basic object types. It obey NameDatabase code as well as AstroObject. NameDatabase was introduced to PlanetarySystem as names index implementation. Names listing display in Celestia view have been simplified and unified.

ToDo: rewrite SolarSystemCatalog to AstroDatabase + DataLoader form.

Existing problems:
There is a hack in NameInfo which may lead to problems in the future. Early its content was set during construction and was immutable. But due to enormously long loading time I was forced to delay initiation of localized name version. I did it by reimplementation of getLocalized() method, which called for the first time, initiate localized name by calling (d)gettext. It generated lots of problems concerning c++ const behaviour. Early I was rewriting many function arguments to non-const, but later I decided to force getLocalized method to be const by casting this. So there is possible flaw in design. Moreover, until getLocalized is called, localized names are unavaliable in search console.
Can anyone suggest a better sollution?

onetwothree
Developer
Posts: 256
Joined: 22.09.2018
With us: 11 months

Post #133by onetwothree » 10.07.2019, 08:29

You can load translations in a separate thread.

Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #134by Lafuente_Astronomy » 14.07.2019, 05:14

pirogronian wrote:Existing problems:
There is a hack in NameInfo which may lead to problems in the future. Early its content was set during construction and was immutable. But due to enormously long loading time I was forced to delay initiation of localized name version. I did it by reimplementation of getLocalized() method, which called for the first time, initiate localized name by calling (d)gettext. It generated lots of problems concerning c++ const behaviour. Early I was rewriting many function arguments to non-const, but later I decided to force getLocalized method to be const by casting this. So there is possible flaw in design. Moreover, until getLocalized is called, localized names are unavaliable in search console.
Can anyone suggest a better sollution?

Specifically what potential future problems would this hack cause? Just curious
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #135by pirogronian » 14.07.2019, 14:17

Lafuente_Astronomy wrote:Specifically what potential future problems would this hack cause? Just curious

More hacks in the code make it more unclear.
Moreover, as I mentioned already, localized names would be unavaliable in search console until getLocalized() is called, what isn't quaranteed at all.

So, I'm preparing my code for thread solution, as @onetwothree suggested.

Avatar
Lafuente_Astronomy
Moderator
Posts: 317
Joined: 04.08.2018
Age: 21
With us: 1 year
Location: Cebu, Philippines
Contact:

Post #136by Lafuente_Astronomy » 14.07.2019, 14:45

pirogronian wrote:More hacks in the code make it more unclear.
Moreover, as I mentioned already, localized names would be unavaliable in search console until getLocalized() is called, what isn't quaranteed at all.

So, I'm preparing my code for thread solution, as @onetwothree suggested.

That's great! I'll be awaiting the code in your next release. Would making the code and successfully removing the hack finish the entire Astro DB Branch or just a part of it?
Official Administrator of the Celestia Discord Server.
Invite: https://discordapp.com/invite/WEWDcJh
If you don't have a Discord account, register here: https://discordapp.com/register
Have a blessed day.

Janus
Posts: 400
Joined: 13.08.2016
With us: 3 years

Post #137by Janus » 14.07.2019, 15:31

Possibly silly question.

Why not do a basic ascii load of names first.
Into a holdervalue called 'simpletext' or something like that.

Then taking advantage of putting everything into one big mess in memory.
Check to if localization is needed.
If it is, get local for that to copy into working labels.
If no localization needed, or available, copy from 'simpletext' to workingtext.

When a search is done, start with simpletext, then onto workingtext, or the other way if desired.
This keeps the core simple, leaving the messy language stuff separate.

Just a silly question.


Janus.

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #138by pirogronian » 15.07.2019, 05:39

Lafuente_Astronomy wrote:Would making the code and successfully removing the hack finish the entire Astro DB Branch or just a part of it?

PlanetarySystem is waiting for full integration.

@Janus
I dont understand everything. Do You sugget startup loading? My goal is fully dynamic loading of data.

@onetwothree
Shared mutex needs at least c++14. May we upgrade project code standard?

Topic author
pirogronian
Developer
Posts: 159
Joined: 05.01.2018
With us: 1 year 7 months
Location: Wrocław

Post #139by pirogronian » 27.07.2019, 21:13

Quick changelog:
Bodies and SolarSystem classes and surrounding infrastructure is integrated with AstroDatabase. Formally it fulfills goals of this branch, but another goal - make objects dynamically unloadable, seems to be at fingertips. So, I'm unsure, what to do. Perhaps it would be good idea to release 1.7 version now and make AstroDB as another major release alone.


Return to “Development”

Who is online

Users browsing this forum: 4 guests