Combine the level of enchantments

I am trying to create an anvil menu that can combine the enchantments of 2 books but the result is a book with the maximum level enchantment. I would like to know how to merge the 2 levels as in a normal anvil

 

You gotta find an algorithm for that.
Enchantments on an item consits of the type and the level stored in a Map. So you have to check/modify two maps.

• You start with book1's enchantments and assume everything is already sorted out (obv it isnt).
• Iterate over book2's enchantments and check that back with book1's enchs (Let's name the current enchantment we're looking at 'currentEnch')

• Check if currentEnch is already in the book1's enchantment map

No: Add it to book1's enchantment map (in addition you can use Enchantment#conflictsWith)
Yes: If the level from the type 'currentEnch' in the book1's map is equal to currentEnch.getMaxLevel() just continue.
If not: Get the currentEnch level in book1 and in book2 and compare it. Take the larger one and store that in book1's map. If they're both equal you increase it by 1 and then store it in book1's map.

An example with that algorithm:
Book1 Enchantments: Sharpness 3, Protection 1, Frostwalker 1, Infinity 1
Book2 Enchantments: Projectile Protection 2, Protection 2, Infinity 1, Sharpness 3

We take Book1 enchantments for granted.
We start looking at book2 enchantments.
Projectile Protection 2 is not in book1 enchantments.
We add Projectile Protection 2 to book1 enchantments.
We look at Protection 2.
Book1 has the type Protection so we compare levels.
Book2's protection (2) > Book1's protection (1) so we store the level 2 in book1's enchantments.
We look at Infinity.
Book1 has the type Infinity.
Book1 Infinity is on max level so we just pass on.
We look at Sharpness 3.
Book1 has the type Sharpness 3 so we compare levels.
They're both the same so we increase it by one and store in book1's enchantments.

We now have the anvil logic used to determine which enchantments stay (except for checking #conflictsWith).
Result: Sharpness 4, Protection 2, Frostwalker 1, Infinity 1, Projectile Protection 2
Hope that points you in the right direction