Abstract This paper describes the use of optical recognition technologies to augment the physical design space and its aesthetic drivers with a myriad of design performance variables available through advanced environmental analysis simulations to create an integrated and collaborative high-performance design platform. By utilizing various hardware and software systems, this design platform can capture critical physical information to be translated into a 3D digital environment where the digital model can be interpreted and various analysis information can be extracted. Once the essential 3D visual information and measured analysis data is generated it can then be displayed back into the design platform in near real-time to be visualized, evaluated, negotiated, and acted upon. This has the greatest benefit to the design process because these decisions are occurring at the moment of design where they have the highest probability of affecting decisions that have the greatest impact on cost, energy use, and overall project design.

Introduction

This paper illustrates the three key points as facets of the design process to be investigated:

1. “Trying to know all things at all times”

Through the visualization and negotiation of multiple data sources simultaneously, a critical and timely understanding the “system” effect of disparate design decisions that can lead to negotiated design outcomes based on measured building performance.

2. Collaborative & Interdisciplinary design platform - “merging non-analysers with non-designers”

Various members of the design team can engage using a common physical platform in which parametric design tools can modify and adapt building designs in response to many different inputs.

3. Real-time design process feedback loop

A real-time feedback loop during the design process merges visual design features with analysis data and metrics used to evaluate the design at the time the building design decisions are being made. This process is critical for a fully-integrated performance based design approach.

Negotiating multiple data sources simultaneously

Physical modelling has been used for a long time as one of the main tools in the architectural design process to negotiate visual design variables such as massing, proportion, and adjacencies. Although the importance of these models cannot be overlooked, they cannot address many of the other conditions included in high performance and environmentally responsible building design. Ultimately, this process tends to limit the design process because of its focus on the finite analysis, and subsequent optimization, of a single design variable and not the combined effect of multiple variables. Once the environmental and formal variables are expanded to include many additional independent sources such as client’s needs (profit, use), governmental needs (codes, zoning), structural (base loads, lateral forces), environmental measures (energy use, daylighting, natural ventilation), we can see how complex the simulation and evaluation of the combined effect of these interrelationships become. In the case of high performance buildings, these interrelationships create very complex systems, with many variables, where single element optimization is not ideal because, at times, the negotiation of disparate variables work against each other.

Disconnected team

The process as described has the possible effect of separating or disconnecting individual project team members into highly specialized disciplines for further analysis and subsequent feedback later in the design process - in many cases this occurs after the design team and client can realize the full benefit of this feedback. In the current professional climate of ever-shorter concept design periods, this older methodology also leads to the use of “rules of thumb” by team members which ignores the often interrelationship and complex interaction of individual design elements. Teams and subsequent designs are not integrated because platform does not enable collaboration, which leads to isolated engineering efforts and post-design analyses.

Design process and analysis feedback loop

Finally, traditional building and environmental simulation or analyses are certainly able to complement building design but are limited due to the fact that it usually requires a long feedback time and a high degree of individual component specificity (i.e. construction types or mechanical systems). Recent workflows are beginning to advance the role of complex environmental simulation and analysis within the early design process by utilizing tools such as Vasari (Autodesk, 2013), Green Building Studio (Autodesk, 2013) , or DIVA (Solemma LLC, 2012) where simulation data can be retrieved and possibly utilized directly by members of the design team. Although these are still individual tools they individually have the effect of speeding up the feedback loop essential to the design process. This improves the chances that the development of integrated design features can be based on simulation data and analytical feedback.

Discussion

Proposed methodologies

A system or platform that analyses simulation data from multiple disciplines or sources and visualizes those simulations in a near real-time feedback loop would allow for a dynamic and collaborative negotiation of these disparate parameters. This paper is demonstrating that this new design paradigm can exist, whereby visualization and synthesis of complex simulation and data analysis can happen simultaneously with the architectural design process while integrating both physical and digital design space. Using real-time model visualization, integrated building metrics, and environmental design simulation within a physical design space, this platform has the potential to dramatically reduce the lag within feedback loop between the performance based simulations and design decision-making. Real design value can then be possible because of the physical collaborative nature of the system, potential speed of various analyses and ultimately the integration of many different data elements resulting in the design of ever more complex and integrated forms of architecture.

One should note that the proposed design platform is not the only method to achieve the desired goals illustrated above. Rather, this type of research ultimately illustrates the notion of designing better tools that enable and support design in the following three ways:

1. Tools that are responsive have the potential of producing more responsive designs through the use of integrated simulation and analysis data.

2. Value propositions require that design parameters have the ability to be measured and, ultimately, the ability of those values to be variable and simultaneously negotiable.

3. Enlightenment through the process of interaction with, or the engagement of, many disparate and complex design variables - the outcome of these interactions is not known and interrelationship cannot be understood without a large amount of simulation or experience.

Project description: Hardware

Currently the design platform is composed of specific hardware components in the following configuration for detection of physical objects and the visualization of the processed / interpreted information. The hardware system consists of three major components: the detection system or “observer”, the assessment and analysis system or “interpreter”, and a system to display results back onto the design space or “visualizer”. These components shown in Figure 1 are discussed in detail here:

Observer

A camera system (Figure 2) is used as the “observer” for object detection and recognition. An infrared band pass filter limits the frequency response of the camera to prevent interference with the visual information that will be projected back on to the detection surface. This detection surface is illuminated by infrared lights tuned to the same frequency as the detection camera to provide a high quality tracking image free of visual spectrum interference that is ideal for object / user detection.

Interpreter

The computer system is the main handler to link the various hardware components and analysis software system into one interface. In the case study we will discuss further, the computer interface is a single high-power computer that creates the pipeline for the optical detection system to connect to the analysis system, run the desired simulations and eventually communicate the interpreted information to the visualization system. The main computational component which is performing the analysis and could be scaled with the complexity of the project and eventually be an array of systems or cloud based servers to increase the computational power of the platform.

Visualizer

A “short-throw” projector (Figure 3) is used to display analysis information onto a piece of high-performance rear projection grade plexi-glass. This setup allows for the digital information to be displayed onto this surface but also onto any physical objects that might be placed on the surface.

Project description: Software

The hardware systems outlined about all tie into a main computer which uses various commercial and custom software which are optimized to interpret, generate, visualize and analyse the incoming video stream for eventual near real-time use by the design team. These various software packages serve to interpret the physical model information, perform the specified analysis, and create the 3D visualization to be re-projected back into the design platform. Each of these tasks is outlined below:

Tracking

Reactivision (Kaltenbrunner, 2009) is an open-source object tracking application software that interprets incoming video information and identifies custom defined “fiducal” markers. These unique markers (Figure 4, 5) are interpreted and translated into a unique object ID with relative X-Y position, and symbol rotation. This information is then sent as a TUIO formatted message (Kaltenbrunner, 2005) over the network utilizing the real-time UDP network protocol (Postel, 1980) to the host machine / machines for later interpretation.

Modelling

Rhinoceros is a 3-Dimensional NURBS surface modelling application that can generate and visualize 3D surface models and is used as the primary interface for the design platform. Grasshopper is a visual scripting plug-in for Rhinoceros that is utilized for its ability as a dynamic parametric modelling tool in which incoming tracking data can be used to position and create geometry within the Rhino 3-D modelling environment as well as perform several different analyses.

Communication

GHowl is an open-source UDP communication interpreter for the grasshopper 3D plug-in. This software component enables two-way communication between the host computers UDP protocol and the grasshopper environment. We use this software to receive the incoming TUIO stream from the Reactivision software as well as sending and receiving OSC formatted messages to and from the iPad reporting interface.

Parametric Model

Within Rhino and specifically Grasshopper the incoming TUIO communication stream can be interpreted and relevant data parsed to signal various outcomes such as the placement of parametric model components, control parametric modifications to those model components or trigger further project measurement and analyses (Figure 6). The nature of these parametric models are dynamic and are ideally suited for an application where models are shaped, moved, and deformed by a user in real-time.

Metrics

Custom definitions are written using native Grasshopper components and are used to determine basic project geometric information such as building dimensions, overall building height, gross floor areas, and geometric relationships such as proximity (Figure 6). Various other geometric metrics have also been tested such as design surface planarity, overall form rationalization, and building component repetition - all of which can be integrated or visualized as needed by the design team through the discussed design interface.

Environmental analysis

Two levels of environmental analyses are currently integrated within the project. For preliminary studies the project uses Ladybug (Sadeghipour, 2013), an open source environmental plug-in for grasshopper that benefits from a simplified fast analysis method to calculate the incident solar radiation on the building envelopes. The result of the analysis can then be tracked and ultimately visualized within the digital model as well as being projected back onto the physical environment augmenting the physical design model and constantly updating results in near real-time.

For advanced energy and daylighting simulation a set of custom Grasshopper components have been developed to automate the exporting of digital Rhino geometry to EnergyPlus (US Department of Energy), Radiance (Ward, 1994) and Daysim (Reinhart and Walkenhorst, 2001). The components use the building massing from the earlier tracked parametric model as the input. Figure 7 shows the scripts that automate the process of intersecting the masses, and finding adjacent surfaces. Floor heights and program of each space could be customized by user. The script subdivides the mass into several zones and assigns construction set, schedules and internal loads for each space based on the program. It also calculates and adds the openings to the geometry based on the percentage of the openings provided by users. The identical geometry is used for both energy and daylighting simulation and the results are read back into

Grasshopper for eventual visualization.

EnergyPlus 7.2 is utilized to run the energy simulations. It executes the simulation on multiple processors that decreases the simulation time and expedites the feedback loop. The user can customize several inputs for each run from the Grasshopper interface such as the percentage of the glazing, running period, construction set, and add or remove shadings to the building. The default output is set to monthly heating, cooling and lighting loads for each zone, however it can be changed to any other EnergyPlus valid outputs.

The result can be tracked and visualized on the screen in the scale of each defined zone or each overall building.

Radiance and Daysim are utilized to run daylighting simulations. Due to the fairly long running time for annual climate-based daylighting simulation, a default recipe for daylighting simulations is set to analyse the daylighting for the 21st of December at 12:00 under an intermediate sky with sun. The material properties are generated based on the EnergyPlus materials for each simulation with the ability for the user to overwrite these values. The user sets the sample grid size and the distance of the points from the base surface - with the default grid size set to 4 meters and the illuminance values measured for points at the height of 0.72 meters high from the floor. The result of the study can be then be visualized for each sample point, an average of each floor or the total average for the building.

Because all other aspects of the proposed design interface are almost instantaneous it is worth noting that the time of this particular simulation varies based on the complexity of geometries and computation power. In the project described in this paper the two project towers and context are evaluated using a local machine with 24 CPUs and computes the result for energy and daylight simulation in approximately 3:00min. Preliminary tests have connected this design platform to cloud computing servers such as JESS (JESS, 2013) which can effectively reduce the calculation time. JESS is a software tool that runs EnergyPlus simulations on remote servers and is currently under development.

Visualization and Reporting

In addition to projecting the visual metrics and analysis information back onto the physical model, the system as proposed also utilizes a tablet device as a reporting tool giving the user a real-time display overview of the design parameters evaluated. As the data is analysed it reports the relevant variables as an OSC formatted message over the UDP protocol through the use of the gHowl plugin (Fraguada, 2013) for Grasshopper (McNeel, 2013). Utilizing an application called TouchOSC (Hexler, 2013) for the graphic presentation of the data, the reporting display is graphically integrated into the visualization workflow. This method allows for a real-time tracking of relevant project data in a consolidated and portable format that the designer can use to track various non-visual data sources.

Current capabilities

The following short paragraphs discuss the current capabilities of the design platform as applied to a fictional project in existing dense urban environment. The site for this project is Chicago, Illinois, USA in the central business district adjacent to the Chicago River, an existing context of tall buildings, and various site features / amenities. A walk-through of a possible scenario and design process will highlight the various design parameter negotiations that are taking place in “real-time”.

Conclusion

This proposed design platform and methodology demonstrates the effect that a tangible design interface has in reshaping current workflows. As discussed earlier, the feedback speed of these tools create a critical iterative design space in which the many and often opposing design variables are visualized and negotiated during the design process. Architectural design is a negotiation and a synthesis of often very disparate and complex data sets from an increasingly wider set of interdisciplinary team members and sources all of which are best exploited when they are fully engaged. When fully engaged, these members can make intelligent value propositions in which design decisions can be measured, understood, and ultimately acted upon in a manner set forth by broad design project goals. The processes and methodologies employed by architects and designers can then be as / more important than the final design product. We have the opportunity to design and create these methodologies and the proposed solution illustrates how important the creation of these tools is when defining an architecture that is responsive to its environment as well as its broader ecosystem.

“...technology can make things faster [and prettier] but not smarter” (Grove, A.)

Acknowledgement

We would like to thank the many talented people that we have the opportunity to collaborate with on a daily basis especially Mario Romero whose assistance and input we would not be able to do without. We would also like to thank those who contribute to the large ecosystem of the open source designers and programmers who also made this project possible.

References

Autodesk, Green Building Studio, http://www.autodesk.com/

Autodesk, Vasari, http://autodeskvasari.com/,

Fraguada, L., gHowl, http://www.grasshopper3d.com/group/ghowl

Hexler, touchOSC, http://hexler.net/software/touchosc

JESS: The jEPlus Simulation Server, De Montfort University, Leicester, United Kingdom, http://www.iesd.dmu.ac.uk/~jeplus/wiki/doku.ph p?id=docs:jess_service

Kaltenbrunner, M. & Bovermann, T. & Bencina, R., Reactivision, http://reactivision.sourceforge.net

Kaltenbrunner, M. & Bovermann, T. & Bencina, R. & Costanza, E. "TUIO - A Protocol for

Table-Top Tangible User Interfaces", Proceedings of the 6th International Workshop on Gesture in Human-Computer Interaction and Simulation (GW 2005), Vannes (France)

McNeel, Robert and Associates, Rhinoceros, http://www.rhino3d.com

McNeel, Robert and Associates, Grasshopper3D, http://www.grasshopper3d.com

Postel, J., 1980, “RFC 768: User Datagram Protocol: Internet Engineering Task Force”, USC / Information Sciences Institute.

Reinhart, C. F., Walkenhorst, O., 2001, “Dynamic RADIANCE-based daylight simulations for a full-scale test office with outer venetian blinds.” Energy & Buildings, 33:7 pp. 683-697.

Sadeghipour Roudsari, Mostapha, Ladybug, http://www.grasshopper3d.com/group/ladybug

Solemma, DIVA, http://www.solemma.net/

US Department of Energy, EnergyPlus: whole building energy simulation program, http://apps1.eere.energy.gov/buildings/energyplus

Ward, G.J. 1994. The RADIANCE Lighting Simulation and Rendering System. Proceedings of the 21st Annual Conference on Computer graphics and interactive techniques, Orlando