Add DGMM in BLAS?

[amontoison]

I was working on an implementation of LBFGS last week in MadNLP (nonlinear optimization solver in Julia) and I discovered that DGMM was only available on GPU (CUBLAS and rocBLAS).
For the implementation of LBFGS or just row / column scaling, it will be nice to have an implementation here in Reference-LAPACK.

DGMM performs A * D or D * A depending on a char side.
A is a dense matrix and D is the diagonal matrix where the diagonal coefficients are provided as a vector d.

[mgates3]

It's also in Intel oneMKL as a batched routine. Here's a link to the cuBLAS version.

The dgmm name is really confusing — dg is not a matrix type in BLAS or LAPACK. To me, looks like a typo of dgemm. Probably should be dimm, that is, diagonal matrix multiply. See Naming Scheme in LAPACK Users Guide.

In SLATE, we added a similar routine called scale_row_col, which does in-place A := R A C for diagonal matrices R and C, used for equilibration. Either R or C can be the identity. (Name probably should be plural, scale_rows_cols, for clarity.) It's helpful to be able to apply both simultaneously, to avoid reading & writing the matrix twice.

Similar routines in LAPACK are laq{ge, gb, he, hb, hp, sy, sb, sp}, which equilibrate general, general-banded, Hermitian, Hermitian-banded, Hermitian-packed, symmetric, symmetric-banded, and symmetric-packed matrices, respectively. Those have logic to decide whether to do the row or column scaling based on cond( R ), cond( C ), or max-norm( A ), it appears. With appropriately chosen rowcnd, colcnd, and Amax, one could force row or column scaling, or both, but that's a pretty backwards way of using the API.