Add server connection, Archive maker and Keyboard navigation#292
Add server connection, Archive maker and Keyboard navigation#292SegerEnd wants to merge 6 commits intoriperiperi:archive-experimentfrom
Conversation
SegerEnd
force-pushed
the
server-is-back
branch
from
a040914 to
843f1f0
Compare
There was a problem hiding this comment.
Glad to see someone picking up on the support for the regular server, since I've been totally invested in getting the archive stuff moving. The improved focusable UI support will definitely help a lot with all the new configuration menus, and the Direct Control mode which requires focus switching a lot.
While the tool for building an archive from a database is useful (and it looks fantastic), the C# migration tools aren't complete yet, and are missing a lot of features (and documentation about what those features do and how to configure them) mostly relating to opening lot saves, something you can't exactly do from python. If this is added now, it will likely need to be updated to match future changes to the archive database structure or migration process. I will try to make it clear when this is fully locked-in.
Specifcally - there's an automated process that will delete sign data that's inaccessible, or on a property that is ban-all or admit-list (there's a human-in-the-loop version of this i'm using to manually make exceptions, too). There's another tool that scans properties for a large difference between the newest and oldest backup, and prefers the oldest backup if it has considerably more objects from ex roommates (useful for counteracting the last save of a property being due to a move-out). I guess these tools will work on the database post-conversion, but it's not really set in stone yet.
The database shape might still change a little, too. There are some things on my TODO list that I don't think I put in the archive database generator:
- Anonymize fso votes
- Remove ban/admit list
Some notes preventing a merge:
- There are a number of changes to the VM and loading process that seem unrelated to anything here and will just mess with things that are either carefully tuned or partially complete (hollow lot reloading).
- There are some changes related to sqlite that might break compatibility with the archive data (specifically forcing UTF8 encoding). Is there a reason for them?
- AllOpenable option on the regular server. I explain more in one of the comments on the code, but it's designed for the more laid-back, freeform archive mode. Without modification, it's not really appropriate for an MMO server, as it's easy for users to be malicious with it.
- I think the behaviour where you can close dialogs with ESC can break the game in a lot of ways...
Job lot desync: Moved LotActive.Set() and ActiveYet = true before the job lot
early-return path in LotContainer, so job lots properly signal readiness instead of hanging
I'm not really seeing this - I recently changed this to fix a desync caused by the new tighter loading process. Did it get lost in translation, or is it no longer an issue?
SegerEnd
force-pushed
the
server-is-back
branch
3 times, most recently
from
5d71e35 to
700301e
Compare
|
+1 docker set up. This will make it easier for people to launch their own FreeSO instances. |
SegerEnd
force-pushed
the
server-is-back
branch
3 times, most recently
from
7770894 to
2074063
Compare
I agree, it was just a quick script for me to try and get a “blank” Archive server running for testing. It wasn’t meant to be a long-term solution. Python isn’t really my go-to :) just spin something quick up without external dependencies. I removed it from the PR. For ones like me that also find it useful for testing, currently you can use it for some time with this command directly from the CLI in the FreeSO/docker directory: curl -sLf https://raw.githubusercontent.com/SegerEnd/FreeSO/tools/docker/fso-archive.py | python3 |
SegerEnd
force-pushed
the
server-is-back
branch
from
2074063 to
71256fe
Compare
|
I am currently working on a Spectator mode for online servers to replace the Its currently working good and in playable state, with both server-side and client-side restrictions in place, i'm going to do some code cleanup of my changes and fix the archive mode that is using currently opening all lots as a spectator (its currently not looking at the archive configuration). Since I’m getting familiar with most parts of the overall architecture, I dont know if everything has been implemented at the appropriate places. I’d appreciate any feedback or suggestions for improvement. Edit: When a admin/lot owner/roommate joins the lot, the lot smoothly transitions into a normal host lot and players can interact with objects again. Without ejecting current spectators on the lot. |
|
Whats spectator gameplay idea? No sim selected or interactions done? |
|
Looks pretty promising, at a glance the changes seem to be in the right places. The transition to becoming a real lot might be one to be careful about - like someone's fish and plants could be killed by someone staying in the lot as a spectator for a long time, then saved for real once the owner joins. My idea for this was originally just fully closing and reopening the lot, but on further examination you could literally reload the lot without disconnecting anyone (the game already does this for a process called "self resync"), though it might be tricky.
You might be able to go farther and try to place them in the positions they were originally at, and try to restore things like their direct control state. It gets more complicated if you want to keep interactions and go-here (VM stack frames and interactions), as deleted objects could reappear when reloading the save, and it will mean the IDs for those objects won't be available. It's an unlikely scenario so it's probably fine for the transition to be a little rough and ignore these things. None of this is necessary for an initial implementation, though. I think leaving the "stayed on the lot for too long, then the owner showed up" case alone would be enough for a first version. The full close and reopen might still be required for joining the purchased version of a lot, as "empty" lots in free roam have a fake ID, reduced terrain generation complexity and no attachment to the nfs/database. I kind of needed to think of this scenario for archive mode anyways. |
|
Its similar to Ts1 house without family and buy / build but without allowing to do that? |
|
Are you happy with how things are right now (including spectator mode)? I can review again - I've been doing some local work that'll conflict, but I can stash it and merge this first if everything is good. |
Yes it’s ready. The only feature not fully implemented yet is saving with dead plants and fish as you mentioned when an owner joins the lot with spectators. otherwise, I’m happy with it. |
|
Release version would be good to test the changes |
riperiperi
left a comment
riperiperi
left a comment
There was a problem hiding this comment.
Left some comments. Looks good, but I've got a few comments on the rules for spectator mode. We can still do a followup that implements the lot reload separate from this PR.
|
|
||
| // For some reason, the sqlite connection sees the value as unsigned. | ||
| return (sbyte)Convert.ToByte(value, CultureInfo.InvariantCulture); | ||
| return (sbyte)Convert.ToInt64(value, CultureInfo.InvariantCulture); |
There was a problem hiding this comment.
Were you encountering an issue with this? I guess maybe it will error for a negative byte, but I never saw this happen and don't know what column could ever trigger it.
There was a problem hiding this comment.
Yes I was, I could not buy a new empty lot as a new sim in Archive mode.
| outIO.Write(size + 14); //size, plus header | ||
| outIO.Write(0); | ||
| outIO.Write(14); | ||
| outIO.Write(62); |
There was a problem hiding this comment.
Why is this 62 now? This should be the offset to the pixel data, which appears right after the header. (14 bytes)
There was a problem hiding this comment.
I got en error at 14 bytes on both MacOS and Linux, It’s for Cursor support on Linux and MacOS, on windows it still works too. I don’t know all the correct offsets, but this seems to work correctly?
I found this online while searching this issue:
- 14 (file header)
- + 40 (BITMAPINFOHEADER)
- + 8 (palette data)
- = 62
I have now improved it by calculating the offset in the method instead of the magic number.
| public LinkedList<VMWalkableRect> Route(Point from, Point to, int startCardinal) | ||
| { | ||
| var openSet = new List<VMWalkableRect>(); | ||
| var openSet = new PriorityQueue<VMWalkableRect, int>(); |
There was a problem hiding this comment.
Not a relevant change for this PR, but it's definitely an improvement.
| ((VMTSOAvatarState)ava.TSOState).Flags |= VMTSOAvatarFlags.Spectator; | ||
| } | ||
|
|
||
| // Self-resync reloads the VM and SyncAllClients pushes the updated state to all clients |
There was a problem hiding this comment.
Self-resync reloads the server VM, which isn't necessary. It's a mechanism to prevent the server from getting into a state that's impossible for clients to sync with. (eg stuck collision... mostly bugs)
I mentioned it earlier as being similar to what we want for the transition from spectator mode, where we reload the lot in-place (which is more like ResetVM, but we want to keep the avatars)
Sending sync to all clients should be enough to update this state without a specialized command, so long as the save is made right now and it doesn't use a cached one.
| if (!IsSpectatorMode) SaveRing(); | ||
| LotSaveTicker = LOT_SAVE_PERIOD; | ||
|
|
||
| Host.UpdateActiveVisitRecords(); |
There was a problem hiding this comment.
This probably shouldn't count in spectator mode (the roomies shouldn't earn visitor hours when they aren't hosting). RecordStartVisit should also probably not fire until the avatar becomes non-spectator, and the lotvisit cleanup at the bottom of ReleaseAvatarClaim should happen if the avatar becomes spectator (and set the visitId to null).
| while (lotActions.Count > 0) lotActions.Dequeue()(); | ||
| } | ||
|
|
||
| if (ShouldTransitionToSpectator) |
There was a problem hiding this comment.
There's quite a bit hiding in this call, and it'll check every tick. I think instead it should check (!AllowGuestOpening || IsArchiveServer || IsSpectatorMode) and transition when it would normally try to close the lot due to noRoomies (after a timeout, not instantly).
Honestly, I'm not sure the transition to spectator can work without a bunch of additional work. Consider the following:
- The host leaves, and 20 players are on the lot using skill objects indoors.
- If we leave them there and enter spectator mode, they're trapped inside, and they're in an interaction when they shouldn't be.
- We'd need to forcibly cancel their interaction then teleport them outside.
- The server also forcibly cancels interactions to remove people from lots, or remove them from saves during ResetVM.
It might be best just to boot everyone in this case for now (as usual) and force them to rejoin, but honestly it's a bit of a similar case to reloading when the lot leaves spectator mode, so solving that could be a path to solving both.
| isRoommate = roomies.Any(r => r.is_pending == 0 && r.avatar_id == avatarId); | ||
|
|
||
| // Spectators still respect ban rules | ||
| if (!isRoommate && !isAdmin |
There was a problem hiding this comment.
My view on this is that the ban rules aren't too important for spectators if the following things are present:
- Players can't interact with objects on the property (done)
- When a roomie joins, the players that are not on the admit list or are on the ban list should be instantly ejected (not done)
- The lot state is reloaded so the player can't have done any damage by waiting for a long time. (not done)
It's probably alright to leave it respecting these rules for now, though it will make the free roam experience a bit painful if they run into a ban_all property while running around, since it'll just suddenly boot them to map screen.
Right now though, this doesn't respect "admit list", which should prevent people from joining a lot unless they are specifically allowed. You can break into one of these properties as things are by hanging around until the owner arrives.
SegerEnd
force-pushed
the
server-is-back
branch
4 times, most recently
from
bec056d to
2c2b6a1
Compare
Get the server connection working again that was lost by the Archive mode changes. Key changes: - Fix server connection bugs caused by the new Archive mode - Update Microsoft.Data.Sqlite.Core to 10.0.2 - Add allOpenable config flag for free roam lot transitions for normal servers. - Add city and lot server configuration options - Make UCP.InitArchive public for CoreGameScreenController - Archive UI improvements (join dialog with direct server IP join)
Add full keyboard and Tab navigation support to the UI framework. Key changes: - Tab/Shift+Tab navigation between focusable UI elements - Keyboard support for UIButton, UISlider, UIListBox, UICombobox - UIRadioButton arrow key navigation within groups - Mouse wheel scrolling for UIListBox - UIContextMenu keyboard navigation (arrow keys, Enter, Escape) - UIGridViewer keyboard support - Focus management via InputManager with IFocusableUI interface
Add FSO.Unix project for native Linux and macOS support. Key changes: - New FSO.Unix project with cross-platform entry point - macOS .app bundle creation and deploy script for macOS/Linux - Native dialogs via osascript (macOS) and zenity (Linux) - ImageSharp-based bitmap/PNG handling instead of Windows drawing library for non-Windows platforms - GameLocator path fixes for TSO on macOS and Linux
Add Docker configuration files for running a FreeSO server. Key changes: - Dockerfile and docker-compose.yml for server deployment - Entrypoint script with secret generation - OCI image labels for container metadata - Default server configuration template (config.json)
SegerEnd
force-pushed
the
server-is-back
branch
from
2c2b6a1 to
1bf330a
Compare
Allow non-roommate visitors to open and explore lots in a read-only spectator mode. Spectators can walk around and observe but cannot modify the lot (build, buy, delete objects, or use most interactions). Key features: - Basic spectator mode for non-roommate lot opening - Block VM commands, cross-room routing, and portal interactions for spectators - Block pie menu on lot objects with error feedback - Spectator-to-writable transition when a roommate/owner joins - Reload lot save and transition back when lot owner joins - Walls up with roof on spectator lot entry - Spectator label shown in UI - Admins excluded from spectator mode restrictions - Chat event when lot transitions from spectator mode - Allow spectators to use direct control - No spectator lots in Archive mode Also includes: cursor fixes for macOS/Linux, PriorityQueue pathfinder optimization, UIListBox focus fix, CurLoader BMP header fix, API config cleanup, and various other minor fixes.
SegerEnd
force-pushed
the
server-is-back
branch
from
1bf330a to
5622e3c
Compare
|
I’ve now added the functionality where when the owner or roommate joins a lot the lot gets reloaded from the previous save. And additionally, users inside a room are automatically returned to the mailbox outside. |
|
Hi! Is everything okay with the current state? Or do I still need to do some changes for approval? |
|
I haven't had much time to work on the archive stuff, so I haven't been able to get around to doing a review. I'll try have another look ASAP, sorry! |
| { | ||
| if (IsSpectator(caller) && vm.GetObjectById(CalleeID) is VMGameObject obj | ||
| && obj.Object.OBJ.GUID != PAYPHONE_GUID && obj.Object.OBJ.GUID != NHOOD_PAYPHONE_GUID) return false; | ||
| if (caller == null && FromNet) return false; |
There was a problem hiding this comment.
This check should happen first, as it eliminates invalid commands. It won't make any functional difference though, so it's fine for now.
riperiperi
left a comment
riperiperi
left a comment
There was a problem hiding this comment.
Looking better, some changes to how the reload is done and I think everything should be covered. I'll rebase my recent changes on top after this PR is merged, no need to worry about merge conflicts there.
Apologies again for the long review delay, I've been busy with other things and it takes a while to get into the proper headspace to review the lot container code (because it's a little bit of a mess of interconnecting checks and processes).
| if (AllowGuestOpening && !ArchiveFreeRoam) | ||
| { | ||
| Host.Broadcast(null, new FSOVMProtocolMessage(true, "21", "22")); | ||
| TransitionToSpectatorMode(); |
There was a problem hiding this comment.
Will this force community or admit mode 5/6 lots to transition to spectator? They're explicitly fully openable by guests, so shouldn't even try to shut down unless nobody is online. That's why the check overwrote noRoomies.
Also, not sure if this will cause unusual behaviour with archive free roam, where it will always run the shutdown timer and as soon as the last person leaves it will instantly close.
| marshal.Deserialize(file); | ||
|
|
||
| Lot.Load(marshal); | ||
| ResyncTime(); |
There was a problem hiding this comment.
I think it needs to do more than this. When loading a save, there are a number of things that might need to be corrected:
- might need to send back objects from avatars who aren't roomies
- move_flags mean the terrain needs to be regenerated, or the lot might even be entirely reset
- Note that this means that lots opened in spectator mode shouldn't consume the move_flags and set them to 0. I've kind of taken a different approach with the archive flags in my new branch, but it wasn't backported to the normal flags.
- Everything in Lot.TSOState might be out of date if the lot was renamed, moved etc.
- Tuning vars from server
- Entity reset; the save might have happened with avatars still on the lot, so they have to be removed (may need to simulate a bunch of ticks to safely cancel their interactions, this happens in
AttemptLoadRing,CleanLot) and object state might have to be patched. - Various lot/object state updates
For this reason, it's probably best that almost all of ResetVM is run again. There are some things that should be skipped:
- Creating a new VM and server driver. That means everything up to and including
Lot.Init();. Maybe this could be moved to a separate function or made optional with an argument likereuseVM = true. - Reloading the HollowLots, that would just be wasted time.
This would also mean there's no need to duplicate the ring loading behavior here.
| if (mailbox != null && ava.Position != LotTilePos.OUT_OF_WORLD | ||
| && !Lot.Context.RoomInfo[Lot.Context.GetRoomAt(ava.Position)].Room.IsOutside) | ||
| { | ||
| SimAntics.Primitives.VMFindLocationFor.FindLocationFor(ava, mailbox, Lot.Context, VMPlaceRequestFlags.UserPlacement); |
There was a problem hiding this comment.
This is unsafe - interaction cancellation can happen over a few seconds, and the interaction script may assume that they're in a slot or specific position. For example, this could instantly eject a sim from a shower while they're still in the showering state, then the script will continue and try to place them outside the shower, indoors (or crash because it expected the sim to be contained).
The safest way to do this right now is via CleanLot() then Lot.Reset(), which will remove all sims from the lot, then you can re-add them to the VM similar to how you're doing it when transitioning from spectator mode. This does mean that sims outside don't retain their interactions or positions, but it avoids any unsafe scenarios. They can't interact with objects outside in spectator mode anyways, so not too much lost for now.
There was a problem hiding this comment.
You might need a way to tell the server driver not to eject players when their sims are deleted by CleanLot().
There was a problem hiding this comment.
Deleted sim disconnection happens in HandleClients(), it'll run a few times when running CleanLot or any other function that calls Lot.Tick. This also processes client commands and inactivity timer, which won't be too useful when trying to reset the lot ASAP. Might be best to add a boolean that disables running HandleClients at all when it's transitioning.
|
Is this ready for review? I notice that a few things changed but not sure if you're entirely done yet. |
|
Yes, I believe it is complete. I have not been working on it recently, but I was having trouble getting some of your recent advice to work without causing the player to return to the city screen. |
4 participants
This PR adds support for connecting to a normal online server alongside Archive mode. You can now choose whether to connect to a server or an archive server from the same UI.
It also adds full keyboard navigation to the UI, improves cross-platform support for macOS and Linux.
Server connection
Archive Maker
Not included in the PR anymore but placed separate, currently you can run instead:
curl -sLf https://raw.githubusercontent.com/SegerEnd/FreeSO/tools/docker/fso-archive.py | python3Docker Support (FreeSO Servers)
Added Docker support for running standard (online) FreeSO servers
Provides a consistent, reproducible environment for development and deployment
This makes it much easier to spin up, test, and deploy normal FreeSO servers without much setup.
Bug fixes
"UPDATE fso_lots l"was missing a trailingspace before
SET, causing SQL errors on neighborhood updates, e.g while purchasing a lot on a server.Config.ArchiveinCityServer.ArchiveHandshake()so non-archive servers work.ShowIPButton?.Positiontoprevent crash when the button doesn't exist. While pressing the "info" icon next to a lot name in Archive offline mode.
System.Data.SQLitewithMicrosoft.Data.Sqlite(10.0.2) across the entire server database layer for cross-platform compatibility. System.Data.SQLite is only Windows compatible. Some projects were already using
Microsoft.Data.SqliteGotoCAS()always forcing archive mode (controller.Archive = truewas hardcoded) breaking servers form functioning.Keyboard navigation
Adds keyboard navigation to the UI. You can now Tab/Shift+Tab between controls, use arrow keys and Enter/Space to interact, and press ESC to close dialogs.
How Tab navigation works
TabIndexif setControls with keyboard support
Native macOS and Linux support
~/Documents/Documents