To model or not to model…thoughts on building energy modeling software.

The home performance field bases itself on what it calls ‘building science’ and its results are supposed to be measurable, quantifiable, repeatable, and numerical.  We test in to a project when we start, we do whatever energy related improvements, and we test out at the end.  And the numbers are supposed to prove both the theory behind our choices and the practice of our actual improvements. 

Last year, I paid several hundred dollars for a software package that models a building’s energy use and then projects it’s use after improvements – and several hundred more for a class to supposedly teach and certify me to use it – and it wasn’t a couple of hours in to the class I was both lost and confused, and also floored at the relative ease with which small changes to my input caused large changes in the output.  The software felt like a black box: I put numbers in here and results came out there and I had noooo idea what relationship the latter had to the former.  Then I’d put in other very similar numbers and whoa, why is that result so different?

I have to say, it WAS partly the software: it was freshly on the market, I was in the first class to try it out, it wasn’t even going to be approved for use for another few months, and even the instructor was pretty lost.  And he’d been using the previous version for years.  OK.  I wasn’t a total numbskull.  But only now, the better part of a year after that day and after many hours of puzzling with this program, finally, today, after a real class on the software from one of the guys who actually *wrote* the software (plus they’re now on version 23), do I finally feel like I’m getting a handle on it.

OK.  So now, even though I think I can use this software to generate better numbers more reflective of the actual home I’m actually modeling, I also now see how easy it is to fudge or subtly shift numbers around to move the output around.  Mmm, all of a sudden I feel like I’m playing Moral Scruples. 

Of course, there are checks on the system.  My final numbers are sent in to a central database overseen by my certifying agency.  And a certain number of my projects are actually visited by another unknown auditor who checks my test numbers.  He goes there, interviews the person I did the work for, and verifies my findings.  And any pattern of strangeness found in my numbers causes me to be subject to even closer scrutiny.  So there’s someone looking over my shoulder, even though from afar.

I’m not alone here; I’ve heard others in my field say that they could make the software say pretty much whatever they wanted.  I guess what I’m grasping at here is that on one hand I’m amazed at the complexity of modeling software systems, convinced of the absolute necessity of them, and humbled by the trust placed in my honesty and integrity by the system.  The whole concept of home performance rests on our ability to quantify what we’ve done.  Without it, there’d be no home performance/energy auditing field. 

And I know how fragile trust is and how hard it is to regain it after it’s lost. 

Whew.  So, when’s the next class in this software?  I need to get better at knowing what I’m doing.  I can’t afford any mistakes.