inversion performance improvements

[sleepybishop]

Some ideas to improve performance of precode matrix inversion:

• During "First Phase", you will find that r == 2 the vast majority of the time, optimizing for this and r <= 3 will improve performance.
• Given enough overhead during decoding (eg. rank(gf2_matrix) == L), inversion can stay in gf2 and avoid expensive hdpc operations completely.
• With some care, you can work only on the dense submatrix U from "Second Phase" and onwards.
• If you are caching the schedule of operations or "prebuilt plan", you can leverage it during "Third Phase" of the initial computation to replay the required operations from the plan to create a sparse U upper instead of matrix multiplication as the rfc implies.

Hopefully these are helpful insights.

[cberner]

Thanks for the suggestions! I spent a while optimizing the r == 1 and r == 2 cases, if you see anything specific that looks like it could be improved though I'd love to know!

Ah yes, decoding is an area that I haven't focused on optimizing very much.

With some care, you can work only on the dense submatrix U from "Second Phase" and onwards. Interesting, that sounds like it could improve encoding performance. Can you tell me more about how that works?

[sleepybishop]

With some care, you can work only on the dense submatrix U from "Second Phase" and onwards. Interesting, that sounds like it could improve encoding performance. Can you tell me more about how that works?

After "First Phase", the i x i matrix in the upper left of A is lower triangular, the operations to turn it into the identity matrix are recorded into the plan but not applied to columns outside of U since it's a given that the i x i submatrix will become an identity matrix.

In order to create the submatrix I_u, the columns 0:i within rows i:L need to be zeroed out, again these changes need not be applied to A since it's a given that they will become zero, instead just the columns within U are modified and the operations required are recorded in the plan.

After this all that is left to do is back solving U_upper using I_u, these operations will not create fill-in in A because all columns to the left are zero.

[cberner]

Ah yes, I think I understand your suggestion.

In order to create the submatrix I_u, the columns 0:i within rows i:L need to be zeroed out, again these changes need not be applied to A since it's a given that they will become zero, instead just the columns within U are modified and the operations required are recorded in the plan.

I do this, by ignoring all columns left of i:

raptorq/src/pi_solver.rs

Line 665 in 95b6b5a

self.backwards_elimination(submatrix, temp, temp, size);

After this all that is left to do is back solving U_upper using I_u, these operations will not create fill-in in A because all columns to the left are zero.

Oh yes, I think I see how this works, and I don't have this optimization. Do you know why the RFC recommends performing the Third Phase by doing this?

the matrix X is multiplied with the submatrix of A consisting of the first i rows of A.  
After this operation, the submatrix of A consisting of the
intersection of the first i rows and columns equals to X, whereas the
matrix U_upper is transformed to a sparse form.

I followed the RFC (

raptorq/src/pi_solver.rs

Line 695 in 95b6b5a

self.A.mul_assign_submatrix(&self.X, self.i);

), but I see how your suggestion of just continuing gaussian elimination would be simpler

edit
If I understand your suggestion correctly, you're saying the 3rd, 4th, and 5th phases can all be combined by apply backwards elimination of U_upper using I_u, after the second phase

[sleepybishop]

Do you know why the RFC recommends performing the Third Phase by doing this?

Because converting the i x i submatrix into the identity matrix makes U_upper pretty dense, making back solving slow.

However like i mentioned in my initial comment:

If you are caching the schedule of operations or "prebuilt plan", you can leverage it during "Third Phase" of the initial computation to replay the required operations from the plan to create a sparse U upper instead of matrix multiplication as the rfc implies.

Instead of maintaining the matrix X you can reverse the operations in the plan that involved converting the i x i matrix from LT to identity, then back solve a much sparser matrix and then reapply those same operations to get back to an identity matrix.

This effectively combines 3rd, 4th, and 5th phases as you surmised but without requiring the X matrix at all.

[cberner]

Got it, right. Thanks for explaining that! Hopefully I'll have time, this weekend, to try implementing this and see how much it improves performance :)

[cberner]

@sleepybishop, I started on this today, but after reading through the RFC & my implementation again I'm not sure how to replay the required operations from the plan to create a sparse U upper instead of matrix multiplication. The operations in the plan include row add operations, whereas X is constructed only from swapping rows & cols. Can you explain how you do that step?

[sleepybishop]

Sure, after "First Phase", the i x i matrix is X and was created purely from row and column swaps.

• From here on out each operation that eliminates a column in the LT part of X is stored in the plan as a row add operation, only the columns within U need to actually be computed as the end result is that X will be an identity matrix.

• After all columns in the LT part of X are gone, mark the position in the list of operations in the plan and continue solving for I_u.

• Now U_upper needs to zeroed out but the previous process has created a lot of fill-in, the rfc says multiply A * X which returns A back to the state it was after First Phase, but that is equivalent to undoing the the operations in the plan previously computed.

• Reapply the each operation from the marked position down to the first operation, (eg. if a row[a] was added to row[b], then add row[a] to row[b] again) since row[a] ^ row[b] ^ row[a] = row[b], but again only make the changes in the columns in U because the columns in X are not required, now U_upper is back to the state it was is after First Phase, sparse, but I_u remains and backsolving will be much faster. Note that when replaying an operation, it is recorded into the plan again.

• After U_upper is zeroed out, 'X' needs to be returned to the identity matrix but the steps to do that are already in the plan so play them again from beginning to the marked position, with the difference being that U will not change since all rows in U_upper are zero. So all that's being accomplished is adding more operations to the plan.

These steps avoid the need to track changes to the X matrix and a having to multiply A*X while resulting in the same steps applied to U. Hopefully it makes a bit more sense now.

[cberner]

Ah yes, I think I understand now. Thanks for explaining it in detail! I'll try it out

[cberner]

Just implemented this and got a nice ~15% speedup: #87 . Thanks for the help!

Do you know of any tricks for optimizing the graph selection for r=2 in the first phase? I've already optimized the data structures quite heavily, but that's still the most expensive processing step. I was thinking maybe I could cache the results, but the elimination process of the first phase seems to frequently create new rows with r=2 which invalidates the cached graph.

[sleepybishop]

Just implemented this and got a nice ~15% speedup: #87 . Thanks for the help!

Good Work!

Do you know of any tricks for optimizing the graph selection for r=2 in the first phase? I've already optimized the data structures quite heavily, but that's still the most expensive processing step. I was thinking maybe I could cache the results, but the elimination process of the first phase seems to frequently create new rows with r=2 which invalidates the cached graph.

One optimization is to use the transpose of A as an index to find the rows that need the count of ones adjusted when a column is moved into U. This can be reused in Fourth Phase to speed up backsolving.

Beyond that, it is possible to prune and grow the graph as needed without recomputing it, however graph maintenance is still relatively expensive.

[cberner]

Ah yep, I already keep an index of values by column to speed up those adjustments.

Ya, I already tried doing graph maintenance for each operation, but that turned out to be slower than just recomputing for each r=2 selection.

[cberner]

@sleepybishop one other question, do you know if implementing this part of the RFC: Furthermore, the row operations required for the HDPC rows may be performed for all such rows in one process, by using the algorithm described in Section 5.3.3.3 significantly improves efficiency relative to computing the matrix product MT * GAMMA and materializing the HDPC rows?

After the latest round of optimizations, essentially all of encoding time is spent in three areas, the last of which can't really be optimized further:

1. Computing MT * GAMMA to generate the HDPC rows
2. First phase, primarily constructing the graph for r=2 selection and then sparse row elimination
3. Performing the symbol additions to generate the actual intermediate symbols (this is already heavily optimized with SIMD)

[sleepybishop]

yes, computing G_HDPC via MT * GAMMA is costly both in memory and cpu because GAMMA is a rather large matrix. So computing it piecemeal is much faster, furthermore when decoding if you have enough repair rows you can avoid computing G_HDPC entirely.

Given enough overhead during decoding (eg. rank(gf2_matrix) == L), inversion can stay in gf2 and avoid expensive hdpc operations completely.

[cberner]

Cool, thanks. Ya, I'd like to implement that decoding optimization at some point. Right now I'm trying to finish up optimizing the encode path

[sleepybishop]

First phase, primarily constructing the graph for r=2 selection and then sparse row elimination

the rfc says the intermediate symbols can be calculated via C = (A^^-1)*D and provides the steps detailed in the phases as a relatively efficient means of achieving that. If you are OK with not following the steps in First Phase to the letter, simpler pivoting strategies like Markowitz pivoting can be used at the expense of some extra row xor ops during back solving.