If you give a matrix with exact symbolic or numerical entries,
Mathematica gives the exact inverse. If, on the other hand, some of the entries in your matrix are approximate real numbers, then
Mathematica finds an approximate numerical result.
When you try to invert a matrix with exact numerical entries,
Mathematica can always tell whether or not the matrix is singular. When you invert an approximate numerical matrix,
Mathematica can usually not tell for certain whether or not the matrix is singular: all it can tell is, for example, that the determinant is small compared to the entries of the matrix. When
Mathematica suspects that you are trying to invert a singular numerical matrix, it prints a warning.
If you work with high-precision approximate numbers,
Mathematica will keep track of the precision of matrix inverses that you generate.
Inverse works only on square matrices.
"Advanced Matrix Operations" discusses the function
PseudoInverse, which can also be used with nonsquare matrices.