[Feature] Create bundled and non-bundled (downloadable dependencies/packages) versions

[BobbyESP]

Pull Request: Move to Downloadable Plugins

Description

This pull request addresses the following changes:

• Transition from Pre-packaged Libraries: We are moving away from using pre-packaged libraries in favor of downloadable plugins. This change aims to enhance flexibility, maintainability, and customization options for our project.

Changes Made

• All of Moved to Kotlinx-serialization and updated Gradle and Kotlin #254
• Moved to Gradle 8.1.4
• Moved to Kotlin 1.9.20
• Created a Jetpack Compose app (that can be removed later) for testing. I feel more comfortable with Compose than XML.
• Migrated to Kotlin DSL (Gradle config)

Benefits

• Flexibility: Downloadable plugins allow users to choose and update specific components independently (even if Python and FFmpeg are mandatory).
• Reduced Package Size: This transition helps in minimizing the overall package size (and also apps size that depend on the lib), resulting in faster downloads and reduced resource consumption.
• Easier Updates: Users can easily update individual plugins without waiting for a complete library update.

Testing

• Made just manual tests.

Additional Notes

• None by now

Related Issues

• Python & FFmpeg as downloadable plugins #248

Please review and provide feedback on the proposed changes.

[BobbyESP]

Not yt-dlp but updates of it in commit a71509

[BobbyESP]

The plugins support is almost finished; I have to change some things related to the initPython function and even the FFMpeg and Aria2c modules of the lib, but the YoutbeDL module itself can start to be reviewed.

[BobbyESP]

The initPython function will have to be modified (as well as the other such as the FFmpeg and Aria2c) since the unzip process is already included in the download process. We should probably have to create different functions depending on the flavor

[BobbyESP]

@JunkFood02 Can we talk about how we should handle the libraries downloading? I mean. Should the download function be inside the init one or something like the already implemented in the app demo?

[BobbyESP]

EDIT: FIXED

Seems to be an error at build time because of the app, here is the log.

I think that the targetConfiguration isn't correct since I would say that the target should be "bundled" or "nonbundled" but when using those cases Gradle isn't able to find the library:

implementation project(":library") targetConfiguration("bundledDebugRuntimeElements")

[JunkFood02]

This PR is quite large (4000+ LoC) and hard to review. Could we please split it to smaller and more focused PRs, each addressing a specific topic?

[BobbyESP]

The changes that are necessary for this functionality require all the work put into this PR since it means almost a rewrite of the library. I will still try to adjust the code to reduce the number of lines and changes, but please keep in mind that this proposed separation requires the changes made.

[BobbyESP]

I will start by removing the Jetpack Compose app

[JunkFood02]

The changes that are necessary for this functionality require all the work put into this PR

We can just make a pull request depending on another, which means a pull request could only be merge after all it's dependencies got merged into the target branch. This allows us to gradually applying the changes and review them in smaller chunks, making the process more manageable.

[BobbyESP]

The changes that are necessary for this functionality require all the work put into this PR

We can just make a pull request depending on another, which means a pull request could only be merge after all it's dependencies got merged into the target branch. This allows us to gradually applying the changes and review them in smaller chunks, making the process more manageable.

That's interesting and cool. Can we create those chunks in some way or I have to create different branches with manual changes?

[JunkFood02]

I have to create different branches with manual changes

I'm afraid you'll have to do so by hand since github doesn't support such cool pull requests while gitlab is offering this as a paid feature

You can start with building upon an existing branch. Submit a pull request when you're ready for a new, separate feature and create a new branch based of the current branch for further changes. Just make sure each branch(PR) focus on a single modification or topic for clarity.

Also, when developing on parallel branches simultaneously, avoid modifying the same files. You know what would happen 😈

[JunkFood02]

For example: you can start with the update_library branch

1. migrating to Gradle Kotlin Script
2. update dependencies
3. other independent changes that could be working on at the same time
   • a new compose app for testing
   • replace fastxml with kotlinx.serialization
   • ...
4. decouple executables and dynamic-linked libraries (this PR)

[BobbyESP]

Hahah okay, thanks for the help! I will do this as it should be done, as you told me. I'll keep this PR open for a while, but when I think the other smaller PRs cover everything done in this one, I'll close it.

[gabefair]

The changes that are necessary for this functionality require all the work put into this PR since it means almost a rewrite of the library. I will still try to adjust the code to reduce the number of lines and changes, but please keep in mind that this proposed separation requires the changes made.

Would it be easier if you slowly mainlined this branch instead of PR'ing into main?

[deniscerri]

@BobbyESP would it be possible to just integrate termux's package manager and internally all the app could do is install packages from the package manager? They would be saved into path and execute normally that way 🤔

[BobbyESP]

@BobbyESP would it be possible to just integrate termux's package manager and internally all the app could do is install packages from the package manager? They would be saved into path and execute normally that way 🤔

You mean to directly download from the app the necessary packages to work from the Termux apt repository? I see what you're aiming for but implementing something like that might be too complex in practice. So while it’s technically possible, doing it cleanly and reliably would require a lot of work — maybe more than what it’s worth, but you can play with it.

I came across a library called Chaquopy that might solve many of the issues we've been facing with Python, dependencies, and integration. I haven't tried it yet, but it looks promising — it allows you to use pip directly to install packages, and it automatically handles downloading and managing Python dependencies within your Android project.

[deniscerri]

@BobbyESP The problem with chaquopy is that the packages are managed by them and alot of them are not available. Like curl cffi. Termux is our best bet i guess
Especially when we have to rely on other executables like ffmpeg.

Id love if we could gather up on discord or telegram, talk this through and try to build a system where packages are built through github actions. We code the shell scripts in the repo and the workflows execute them and then the app downloads them.

I introduced the idea to xibr, he mentioned its difficult but i think we can figure it out.

[BobbyESP]

@BobbyESP The problem with chaquopy is that the packages are managed by them and alot of them are not available. Like curl cffi. Termux is our best bet i guess Especially when we have to rely on other executables like ffmpeg.

Id love if we could gather up on discord or telegram, talk this through and try to build a system where packages are built through github actions. We code the shell scripts in the repo and the workflows execute them and then the app downloads them.

I introduced the idea to xibr, he mentioned its difficult but i think we can figure it out.

Hi mate!

Yeah, you're absolutely right. I personally prefer to discuss things over Telegram first, if that works for you. Thanks a lot for reaching out! I'd be happy to help, even though I’m no longer working on this, but it definitely sounds like an exciting challenge.

[deniscerri]

we could make a gc in telegram, invite me