New decompile-free faster build process

[agaricusb]

Adds a new and improved (but experimental) build process as discussed with @AbrarSyed on IRC earlier (just missed you on #minecraftforge today). Removes the need for MCP, decompilation, and excessive recompilation – in favor of a brand new SpecialSource-based remapper currently executed via Maven.

Builds FE in ~7 seconds on my system.

Known issues:

• Localizations are not added (need to find out how to pull from https://github.com/ForgeEssentials/FELocalizations?)
• Versions not substituted in resources (should be easy with resource filtering)
• Client is not compiled (separate project? subproject?)

However, it does produce a usable build (run with 'mvn initialize' then 'mvn package', mod will be in target/). Again it is still experimental, but if FE wants to be a pilot mod for SSMP.. :) Probably needs further testing before pulling (although note both build processes can coexist).

[yuuka-miya]

1. Sure, great, I wanted to do an ant build script, but lazy me would rather pull than write :DD
2. If maven can call shell commands, use sed?
3. FE_SRC_LIBS has to be left alone, there is no need to compile.
4. Yep, client is a separate project.
5. I'm not ENTIRELY sure how SpecialSource will affect our (somewhat excessive, I must say) ASM usage. Care to shed some light on it?
6. I can make this setup run on the beta builds, for testing if needed.
7. And with runtime deobfuscation coming in 1.5, I don't think there'll be a need to use SpecialSource?
8. I also need you to preserve our current directory structure.
9. http://paste.minecraftforge.net/view/c1d45080

[AbrarSyed]

I will try and test out how this is gona go and stuff, and if we can afford to bring it in without changing our structure too much.

[dries007]

I he can show me how to implement this on our jenkins, I'm all for it.

[AbrarSyed]

uh, i guess you can start by installing maven...

[dries007]

If is installed by jenkins I beleve, Or you can do it, you have sudo rights.

[yuuka-miya]

i have set jenkins up already, look in fe-betas-mvnexperimental

[agaricusb]

i have set jenkins up already, look in fe-betas-mvnexperimental

Cool, thanks guys for your willingness to try out this crazy new build technique. There might be some kinks to work out in the process, but if successful for a project as large as FE, I think it won't be long before other mods can adopt it as well. Couple things:

2. If maven can call shell commands, use sed?

It can call out to arbitrary plugins, but Maven has something better built-in, eliminating the need for external processes: resource filtering. ${...} variables will be expanded in directories when true. Still need to find out how to substitute the version in code, however.. CraftBukkit reads the version dynamically from the version.properties in the jar, but I couldn't find out how to get this to work with FML annotations, which required constants only (a final variable initialized in a static initializer was insufficient).

3. FE_SRC_LIBS has to be left alone, there is no need to compile.

Well I couldn't get FE_SRC_COMMON to compile without it.. there's a way to add multiple source directories but it has some possible issues. Another alternative is compiling libs in a separate project then adding it as a dependency.

5. I'm not ENTIRELY sure how SpecialSource will affect our (somewhat excessive, I must say) ASM usage. Care to shed some light on it?

Sure. SS now supports access transformers, so FE compiles against a 'minecraft-forge+forgeessentials' dependency which has your AT's applied. The dependency doesn't have other ASM transformers executed on it, but since its only used for compilation, this should not matter unless you use ASM to add new symbols in net.minecraft (doesn't seem that FE does this) and reference them statically. The ASM transformers still run at runtime. Also the FMLCorePlugin property is added in the manifest with maven-jar-plugin.

6. I can make this setup run on the beta builds, for testing if needed.

Sweet

7. And with runtime deobfuscation coming in 1.5, I don't think there'll be a need to use SpecialSource?

There's still a need to remap to the numeric "srg" names, from the descriptive "csv" names used in MCP. Haven't started looking into 1.5 yet but I expect a fairly similar setup to work (probably with true to avoid reading the .csvs).

8. I also need you to preserve our current directory structure.

Fortunately Maven can handle custom directory configurations, normally source is in src/main/java but I set it to ${basedir}/src/FE_SRC_COMMON; same with resources. Some of the files however couldn't find a reasonable way to fit them in.

9. http://paste.minecraftforge.net/view/c1d45080

Currently these projects are not available on Maven Central; have to be built and installed from source until then.. working on getting this fixed soon

[dries007]

To be faire, I don't care if you have to recompile the libs if you cut down
the time from 10 to under 1 minutes.

[yuuka-miya]

@dries007, what's done is done, actually.
@agaricusb what i meant by directory structure was, have you seen the way the servercomplete package is made?
the servercomplete zip is just "extract into server dir and go", our shell script will actually go and put all the modules in the mods folder and the core in the coremods folder.
and just tell it to compile using the libs in the lib folder in the repo. compiled versions of FE_SRC_LIBS are put there.

[agaricusb]

if you cut down the time from 10 to under 1 minutes.

On my system this process builds FE in 5 seconds =)

what i meant by directory structure was, have you seen the way the servercomplete package is made?
the servercomplete zip is just "extract into server dir and go", our shell script will actually go and put all the modules in the mods folder and the core in the coremods folder.

Ah I see.. hmm. Not sure how to create this "zip of zips" with Maven. I see what Abrar was saying earlier now about how building ForgeEssentials with Gradle may be easier, since it has a more expressive language for specifying the build steps than Maven, and it would probably be easier to convert existing shell scripts, especially for this double-zipping process.

Should be possible nonetheless. Maybe remapping using SpecialSource with Gradle instead of Maven; this would consist of converting the SpecialSourceMP plugin to Gradle. SS is fairly modular so this wouldn't be too diffcult; SSMP is just a thin wrapper around SS. Something to investigate..

I'll close this PR for now; the changes are on https://github.com/ForgeEssentials/ForgeEssentialsMain/tree/experimental. I'll also keep my branch at https://github.com/agaricusb/ForgeEssentialsMain/tree/maven-dev updated with the latest experimental changes as this process is refined; anything you need catch me on IRC.

[yuuka-miya]

Sure.