Finalizers are longer guaranteed to be called on exit now that
we switched to .NET Core. This results in native instances leaking.
The only solution I can think of so far is to keep a list of all
instances alive to dispose when the AssemblyLoadContext.Unloading
event is raised.
We're targeting .NET 5 for now to make development easier while
.NET 6 is not yet released.
TEMPORARY REGRESSIONS
---------------------
Assembly unloading is not implemented yet. As such, many Godot
resources are leaked at exit. This will be re-implemented later
together with assembly hot-reloading.
The main focus here was to remove the majority of code that relied on
Mono's embedding APIs, specially the reflection APIs. The embedding
APIs we still use are the bare minimum we need for things to work.
A lot of code was moved to C#. We no longer deal with any managed
objects (`MonoObject*`, and such) in native code, and all marshaling
is done in C#.
The reason for restructuring the code and move away from embedding APIs
is that once we move to .NET Core, we will be limited by the much more
minimal .NET hosting.
PERFORMANCE REGRESSIONS
-----------------------
Some parts of the code were written with little to no concern about
performance. This includes code that calls into script methods and
accesses script fields, properties and events.
The reason for this is that all of that will be moved to source
generators, so any work prior to that would be a waste of time.
DISABLED FEATURES
-----------------
Some code was removed as it no longer makes sense (or won't make sense
in the future).
Other parts were commented out with `#if 0`s and TODO warnings because
it doesn't make much sense to work on them yet as those parts will
change heavily when we switch to .NET Core but also when we start
introducing source generators.
As such, the following features were disabled temporarily:
- Assembly-reloading (will be done with ALCs in .NET Core).
- Properties/fields exports and script method listing (will be
handled by source generators in the future).
- Exception logging in the editor and stack info for errors.
- Exporting games.
- Building of C# projects. We no longer copy the Godot API assemblies
to the project directory, so MSBuild won't be able to find them. The
idea is to turn them into NuGet packages in the future, which could
also be obtained from local NuGet sources during development.
We will be progressively moving most code to C#.
The plan is to only use Mono's embedding APIs to set things at launch.
This will make it much easier to later support CoreCLR too which
doesn't have rich embedding APIs.
Additionally the code in C# is more maintainable and makes it easier
to implement new features, e.g.: runtime codegen which we could use to
avoid using reflection for marshaling everytime a field, property or
method is accessed.
SOME NOTES ON INTEROP
We make the same assumptions as GDNative about the size of the Godot
structures we use. We take it a bit further by also assuming the layout
of fields in some cases, which is riskier but let's us squeeze out some
performance by avoiding unnecessary managed to native calls.
Code that deals with native structs is less safe than before as there's
no RAII and copy constructors in C#. It's like using the GDNative C API
directly. One has to take special care to free values they own.
Perhaps we could use roslyn analyzers to check this, but I don't know
any that uses attributes to determine what's owned or borrowed.
As to why we maily use pointers for native structs instead of ref/out:
- AFAIK (and confirmed with a benchmark) ref/out are pinned
during P/Invoke calls and that has a cost.
- Native struct fields can't be ref/out in the first place.
- A `using` local can't be passed as ref/out, only `in`. Calling a
method or property on an `in` value makes a silent copy, so we want
to avoid `in`.
REGARDING THE BUILD SYSTEM
There's no longer a `mono_glue=yes/no` SCons options. We no longer
need to build with `mono_glue=no`, generate the glue and then build
again with `mono_glue=yes`. We build only once and generate the glue
(which is in C# now).
However, SCons no longer builds the C# projects for us. Instead one
must run `build_assemblies.py`, e.g.:
```sh
%godot_src_root%/modules/mono/build_scripts/build_assemblies.py \
--godot-output-dir=%godot_src_root%/bin \
--godot-target=release_debug`
```
We could turn this into a custom build target, but I don't know how
to do that with SCons (it's possible with Meson).
OTHER NOTES
Most of the moved code doesn't follow the C# naming convention and
still has the word Mono in the names despite no longer dealing with
Mono's embedding APIs. This is just temporary while transitioning,
to make it easier to understand what was moved where.
Adds a new, cleaned up, HashMap implementation.
* Uses Robin Hood Hashing (https://en.wikipedia.org/wiki/Hash_table#Robin_Hood_hashing).
* Keeps elements in a double linked list for simpler, ordered, iteration.
* Allows keeping iterators for later use in removal (Unlike Map<>, it does not do much
for performance vs keeping the key, but helps replace old code).
* Uses a more modern C++ iterator API, deprecates the old one.
* Supports custom allocator (in case there is a wish to use a paged one).
This class aims to unify all the associative template usage and replace it by this one:
* Map<> (whereas key order does not matter, which is 99% of cases)
* HashMap<>
* OrderedHashMap<>
* OAHashMap<>
`DirAccess *` needs to be deleted manually, and this is often forgotten
especially when doing early returns with `ERR_FAIL_COND`.
`DirAccessRef` is deleted automatically when it goes out of scope.
Co-authored-by: bruvzg <7645683+bruvzg@users.noreply.github.com>
Settings that aren't within a subsection are difficult to reach when
other settings do have a subsection.
This also adds documentation for the project setting.
Sets `AlignOperands` to `DontAlign`.
`clang-format` developers seem to mostly care about space-based indentation and
every other version of clang-format breaks the bad mismatch of tabs and spaces
that it seems to use for operand alignment. So it's better without, so that it
respects our two-tabs `ContinuationIndentWidth`.
The current code style guidelines forbid the use of `auto`.
Some uses of `auto` are still present, such as in UWP code (which
can't be currently tested) and macros (where removing `auto` isn't
easy).
This source generator adds a newly introduced attribute,
`ScriptPath` to all classes that:
- Are top-level classes (not inner/nested).
- Have the `partial` modifier.
- Inherit `Godot.Object`.
- The class name matches the file name.
A build error is thrown if the generator finds a class that meets these
conditions but is not declared `partial`, unless the class is annotated
with the `DisableGodotGenerators` attribute.
We also generate an `AssemblyHasScripts` assembly attribute which Godot
uses to get all the script classes in the assembly, eliminating the need
for Godot to search them. We can also avoid searching in assemblies that
don't have this attribute. This will be good for performance in the
future once we support multiple assemblies with Godot script classes.
This is an example of what the generated code looks like:
```
using Godot;
namespace Foo {
[ScriptPathAttribute("res://Player.cs")]
// Multiple partial declarations are allowed
[ScriptPathAttribute("res://Foo/Player.cs")]
partial class Player {}
}
[assembly:AssemblyHasScripts(new System.Type[] { typeof(Foo.Player) })]
```
The new attributes replace script metadata which we were generating by
determining the namespace of script classes with a very simple parser.
This fixes several issues with the old approach related to parser
errors and conditional compilation.
It also makes the task part of the MSBuild project build, rather than
a separate step executed by the Godot editor.
Happy new year to the wonderful Godot community!
2020 has been a tough year for most of us personally, but a good year for
Godot development nonetheless with a huge amount of work done towards Godot
4.0 and great improvements backported to the long-lived 3.2 branch.
We've had close to 400 contributors to engine code this year, authoring near
7,000 commit! (And that's only for the `master` branch and for the engine code,
there's a lot more when counting docs, demos and other first-party repos.)
Here's to a great year 2021 for all Godot users 🎆
I couldn't find a tool that enforces it, so I went the manual route:
```
find -name "thirdparty" -prune \
-o -name "*.cpp" -o -name "*.h" -o -name "*.m" -o -name "*.mm" \
-o -name "*.glsl" > files
perl -0777 -pi -e 's/\n}\n([^#])/\n}\n\n\1/g' $(cat files)
misc/scripts/fix_style.sh -c
```
This adds a newline after all `}` on the first column, unless they
are followed by `#` (typically `#endif`). This leads to having lots
of places with two lines between function/class definitions, but
clang-format then fixes it as we enforce max one line of separation.
This doesn't fix potential occurrences of function definitions which
are indented (e.g. for a helper class defined in a .cpp), but it's
better than nothing. Also can't be made to run easily on CI/hooks so
we'll have to be careful with new code.
Part of #33027.
Which means that reduz' beloved style which we all became used to
will now be changed automatically to remove the first empty line.
This makes us lean closer to 1TBS (the one true brace style) instead
of hybridating it with some Allman-inspired spacing.
There's still the case of braces around single-statement blocks that
needs to be addressed (but clang-format can't help with that, but
clang-tidy may if we agree about it).
Part of #33027.
Not sure if we should check revision too, but this is good enough for what we want.
This will be needed to load the correct Microsoft.Build when we switch to the nuget version.
- Include PDB files in exported games.
- Release export templates also allow debugging now.
Right now the only way to enable debugging in exported games is with the environment variables, which may be cumbersome or not even possible on some platforms.
Right now, games only work on devices when exported with FullAOT+Interpreter.
There are some issues left that need to addressed for FullAOT alone. Right now,
it's giving issues with the Godot.NativeCalls static constructor.
EngineDebugger is the new interface to access the debugger.
It tries to be as agnostic as possible on the data that various
subsystems can expose.
It allows 2 types of interactions:
- Profilers:
A subsystem can register a profiler, assigning it a unique name.
That name can be used to activate the profiler or add data to it.
The registered profiler can be composed of up to 3 functions:
- Toggle: called when the profiler is activated/deactivated.
- Add: called whenever data is added to the debugger
(via `EngineDebugger::profiler_add_frame_data`)
- Tick: called every frame (during idle), receives frame times.
- Captures: (Only relevant in remote debugger for now)
A subsystem can register a capture, assigning it a unique name.
When receiving a message, the remote debugger will check if it starts
with `[prefix]:` and call the associated capture with name `prefix`.
Port MultiplayerAPI, Servers, Scripts, Visual, Performance to the new
profiler system.
Port SceneDebugger and RemoteDebugger to the new capture system.
The LocalDebugger also uses the new profiler system for scripts
profiling.
Happy new year to the wonderful Godot community!
We're starting a new decade with a well-established, non-profit, free
and open source game engine, and tons of further improvements in the
pipeline from hundreds of contributors.
Godot will keep getting better, and we're looking forward to all the
games that the community will keep developing and releasing with it.
Up until now debug builds would always wait up to 500 ms during initialization
to give time for debuggers to attach to the game.
We no longer want this as it increases startup time unnecesarily.
The way forward is to setup the debugger agent as client instead of server.
This way it's the game that connect to the debugger, not the other way around.
If server mode is still desired, suspend=y can be used to indefinitely wait
for the debugger to attach. This all can be specified with the environment
variable 'GODOT_MONO_DEBUGGER_AGENT' when launching the game.
This appears to be necessary for current official builds cross-compiled
with MinGW from Linux, using Mono 6.6.0.160.
Follow-up to #31784, see #29812 for details.
- Added correct config file for android dllmaps.
- Fix __Internal DllImports with a dlopen fallback.
- Add missing P/Invoke functions and internal calls expected by the monodroid BCL and our custom version of the 'Android.Runtime.AndroidEnvironment' class (this last one can be found in the godot-mono-builds repo).
- Make sure to set 'btls' instead of 'legacy' as the default TLS provider on Android.
API hashes cannot be calculated on release builds, as bindings information is lacking. Therefore, we should not be comparing it with the generated glue hash as they will never match.
This will be used for communicating between the Godot editor and external IDEs/editors, for things like opening files, triggering hot-reload and running the game with a debugger attached.
Previously, when running the project manager, we would try to load the API assemblies from the project and fail because we were not editing any project. This would make us try to copy the prebuilt API assemblies to the project. Since there is no project, it would try to copy them to the executable location. This would fail if Godot doesn't have permissions to write to that location.
This commit fixes that by instead trying to load the prebuilt API assemblies in the first place, if running the project manager.
By default, an unhandled exception will cause the application to be terminated; but the project setting `mono/unhandled_exception_policy` was added to change this behaviour.
The editor is hard-coded to never terminate because of unhandled exceptions, as that would make writing editor plugins a painful task, and we cannot kill the editor because of a mistake in a thirdparty plugin.
If both the core and editor API assemblies are missing or out of sync, Godot will only update the former and then abort when trying to load them again because the latter was not updated. Godot will update it correctly the next time it's started, but this should not be needed and it should work the first time. This commit fixes that.
Remove the old API assembly invalidation system. It's pretty simple since now the editor has a hard dependency on the API assemblies and SCons takes care of prebuilding them.
If we fail to load a project's API assembly because it was either missing or outdated, we just copy the prebuilt assemblies to the project and try again. We also do this when creating the solution and before building, just in case the user removed them from the disk after they were loaded.
This way the API assemblies will be always loaded successfully. If they are not, it's a bug.
Also fixed:
- EditorDef was behaving like GlobalDef in GodotTools.
- NullReferenceException because we can't serialize System.WeakReference yet. Use Godot.WeakRef in the mean time.
Make the build system automatically build the C# Api assemblies to be shipped with the editor.
Make the editor, editor player and debug export templates use Api assemblies built with debug symbols.
Always run MSBuild to build the editor tools and Api assemblies when building Godot.
Several bugs fixed related to assembly hot reloading and restoring state.
Fix StringExtensions internal calls not being registered correctly, resulting in MissingMethodException.
- Only load the scripts metadata file when it's really needed. This way we avoid false errors, when there is no C# project, about missing scripts metadata file.
Apparently we don't need to call mono_debug_close_image ourselves and we can call mono_image_close right away as it's not our duty to keep that reference.
Also fixed a wrong ifdef that was causing Mono to never be initialized if mscorlib was not found (which was the case with the utf8 assemblies path bug this commit fixes).
This condition was meant for exported projects only, not for the editor only.
This is needed to avoid aborting due to missing mscorlib for projects that do not use C#.
If 'res://.mono/' exists, then we assume the project uses C#, in which case a missing mscorlib should still abort.
- Now there is only one solution that contains both GodotSharp and GodotSharpEditor project. Previously we had one solution for each project
- GodotSharpEditor reference GodotShatp with a 'ProjectReference'. Previously it was a 'Reference' to the assembly
- This also simplifies the command line option to generate this solution: 'godot --generate-cs-api <OutputDir>'
Ignacio Roldán Etcheverry
bruvzg
reduz
Rémi Verschelde
Hendrik Brucker
Hugo Locurcio
Max Hilbrunner
Aaron Franke
Lightning_A
Marcel Admiraal
NutmegStudio
lupoDharkael
Fabio Alessandrelli
Ivan Shakhov
Ignacio Roldán Etcheverry
Sebastian Hartte
Carter Anderson
marxin
Mads Ynddal