The way we perform calibration for classification problems and compare the results based on reliability plot, I am wondering if there are methods for regression outcomes, comparing the distribution error?

The term ‘calibration’ has several differing meanings depending on context. In the context of plotting calibration for regression, this is a good practice. Here’s a walk through with a linear regression model from a chemistry perspective: https://chem.libretexts.org/Courses/Providence_College/CHM_331_Advanced_Analytical_Chemistry_1/05%3A_Standardizing_Analytical_Methods/5.04%3A_Linear_Regression_and_Calibration_Curves\

I would recommend that you search for ‘Diagnostic Plots’ for regression models. There’s a lot to choose from and all present different information about the performance of the model.