Monday, January 6th, 2020

Time	Event
12:00a	Tool predicts how fast code will run on a chip MIT researchers have invented a machine-learning tool that predicts how fast computer chips will execute code from various applications.   To get code to run as fast as possible, developers and compilers — programs that translate programming language into machine-readable code — typically use performance models that run the code through a simulation of given chip architectures.  Compilers use that information to automatically optimize code, and developers use it to tackle performance bottlenecks on the microprocessors that will run it. But performance models for machine code are handwritten by a relatively small group of experts and are not properly validated. As a consequence, the simulated performance measurements often deviate from real-life results.  In series of conference papers, the researchers describe a novel machine-learning pipeline that automates this process, making it easier, faster, and more accurate. In a paper presented at the International Conference on Machine Learning in June, the researchers presented Ithemal, a neural-network model that trains on labeled data in the form of “basic blocks” — fundamental snippets of computing instructions — to automatically predict how long it takes a given chip to execute previously unseen basic blocks. Results suggest Ithemal performs far more accurately than traditional hand-tuned models.  Then, at the November IEEE International Symposium on Workload Characterization, the researchers presented a benchmark suite of basic blocks from a variety of domains, including machine learning, compilers, cryptography, and graphics that can be used to validate performance models. They pooled more than 300,000 of the profiled blocks into an open-source dataset called BHive. During their evaluations, Ithemal predicted how fast Intel chips would run code even better than a performance model built by Intel itself.  Ultimately, developers and compilers can use the tool to generate code that runs faster and more efficiently on an ever-growing number of diverse and “black box” chip designs. “Modern computer processors are opaque, horrendously complicated, and difficult to understand. It is also incredibly challenging to write computer code that executes as fast as possible for these processors,” says co-author Michael Carbin, an assistant professor in the Department of Electrical Engineering and Computer Science (EECS) and a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL). “This tool is a big step forward toward fully modeling the performance of these chips for improved efficiency.” Most recently, in a paper presented at the NeurIPS conference in December, the team proposed a new technique to automatically generate compiler optimizations.  Specifically, they automatically generate an algorithm, called Vemal, that converts certain code into vectors, which can be used for parallel computing. Vemal outperforms hand-crafted vectorization algorithms used in the LLVM compiler — a popular compiler used in the industry. Learning from data Designing performance models by hand can be “a black art,” Carbin says. Intel provides extensive documentation of more than 3,000 pages describing its chips’ architectures. But there currently exists only a small group of experts who will build performance models that simulate the execution of code on those architectures.  “Intel’s documents are neither error-free nor complete, and Intel will omit certain things, because it’s proprietary,” Mendis says. “However, when you use data, you don’t need to know the documentation. If there’s something hidden you can learn it directly from the data.” To do so, the researchers clocked the average number of cycles a given microprocessor takes to compute basic block instructions — basically, the sequence of boot-up, execute, and shut down — without human intervention. Automating the process enables rapid profiling of hundreds of thousands or millions of blocks.  Domain-specific architectures In training, the Ithemal model analyzes millions of automatically profiled basic blocks to learn exactly how different chip architectures will execute computation. Importantly, Ithemal takes raw text as input and does not require manually adding features to the input data. In testing, Ithemal can be fed previously unseen basic blocks and a given chip, and will generate a single number indicating how fast the chip will execute that code.  The researchers found Ithemal cut error rates in accuracy — meaning the difference between the predicted speed versus real-world speed — by 50 percent over traditional hand-crafted models. Further, in their next paper, they showed that Ithemal’s error rate was 10 percent, while the Intel performance-prediction model’s error rate was 20 percent on a variety of basic blocks across multiple different domains. The tool now makes it easier to quickly learn performance speeds for any new chip architectures, Mendis says. For instance, domain-specific architectures, such as Google’s new Tensor Processing Unit used specifically for neural networks, are now being built but aren’t widely understood. “If you want to train a model on some new architecture, you just collect more data from that architecture, run it through our profiler, use that information to train Ithemal, and now you have a model that predicts performance,” Mendis says. Next, the researchers are studying methods to make models interpretable. Much of machine learning is a black box, so it’s not really clear why a particular model made its predictions. “Our model is saying it takes a processor, say, 10 cycles to execute a basic block. Now, we’re trying to figure out why,” Carbin says. “That’s a fine level of granularity that would be amazing for these types of tools.” They also hope to use Ithemal to enhance the performance of Vemal even further and achieve better performance automatically. (Comment on this)
3:20p	An algorithm with an eye for visibility helps pilots in Alaska More than three-quarters of Alaskan communities have no access to highways or roads. In these remote regions, small aircraft are a town's bus, ambulance, and food delivery — the only means of getting people and things in and out.  As routine as daily flight may be, it can be dangerous. These small (or general aviation) aircraft are typically flown visually, by a pilot looking out the cockpit windows. If sudden storms or fog appears, a pilot might not be able to see a runway, nearby aircraft, or rising terrain. In 2018, the Federal Aviation Administration (FAA) reported 95 aviation accidents in Alaska, including several fatal crashes that occurred in remote regions where poor visibility may have played a role. "General aviation pilots in Alaska need to be aware of the forecasted conditions during pre-flight planning, but also of any rapidly changing conditions during flight," says Michael Matthews, a meteorologist at MIT Lincoln Laboratory. "There are certain rules, like you can't fly with less than three miles of visibility. If it is worse, pilots need to fly on instruments, but they need to be certified for that." Pilots check current or forecasted weather conditions before they fly, but a lack of automated weather observation stations throughout the Alaskan bush makes it hard to know exactly what to expect. To help, the FAA recently installed 221 web cameras near runways and mountain passes. Pilots can look at the image feeds online to plan their route. Still, it's difficult to go through what could be hundreds of images and estimate just how far one can see. So, Matthews has been working with the FAA to turn these web cameras into visibility sensors. He has developed an algorithm, called Visibility Estimation through Image Analytics (VEIA), that uses a camera's image feed to automatically determine the area's visibility. These estimates can then be shared among forecasters and with pilots online in real-time. Trained eyes In concept, the VEIA algorithm determines visibility the same way humans do. It looks for stationary "edges." For human observers, these edges are landmarks of known distances from an airfield, such as a tower or mountain top. They're trained to interpret how well they can see each marker compared to on a clear, sunny day. Likewise, the algorithm is first taught what edges look like in clear conditions. The system looks at the past 10 days' worth of imagery, an optimal timeframe because any shorter timeframe could be skewed by bad weather and any longer could be affected by seasonal changes, according to Matthews. Using these 10-day images, the system creates a composite "clear" image. This image becomes the reference to which a current image is compared. To run a comparison, an edge-detection algorithm (called a Sobel filter) is applied to both the reference and current image. This algorithm identifies edges that are persistent — the horizon, buildings, mountain sides — and removes fleeting edges like cars and clouds. Then, the system compares the overall edge strengths and generates a ratio. The ratio is converted into visibility in miles. Developing an algorithm that works well across images from any web camera was challenging, Matthews says. Based on where they are placed, some cameras might have a view of 100 miles and others just 100 feet. Other problems stemmed from permanent objects that were very close to the camera and dominated the view, such as a large antenna. The algorithm had to be designed to look past these near objects. "If you're an observer on Mount Washington, you have a trained eye to look for very specific things to get a visibility estimate. Say, the ski lifts on Attitash Mountain, and so on. We didn't want to make an algorithm that is trained so specifically; we wanted this same algorithm to apply anywhere and across all types of edges," Matthews says. To validate its estimates, the VEIA algorithm was tested against data from Automated Surface Observing Stations (ASOS). These stations, of which there are close to 50 in Alaska, are outfitted with sensors that can estimate visibility each hour. The VEIA algorithm, which provides estimates every 10 minutes, was more than 90 percent accurate in detecting low-visibility conditions when compared to co-located ASOS data. Informed pilots The FAA plans to test the VEIA algorithm in summer 2020 on an experimental website. During the test period, pilots can visit the experimental website to see real-time visibility estimates alongside the camera imagery itself. "Furthermore, the VEIA estimates can be ingested into weather prediction models to improve the forecasts," says Jenny Colavito, who is the ceiling and visibility research project lead at the FAA. "All of this leads to keeping pilots better informed of weather conditions so that they can avoid flying into hazards." The FAA is looking into using weather cameras in other regions, starting in Hawaii. "Like Alaska, Hawaii has extreme terrain and weather conditions that can change rapidly. I anticipate that the VEIA algorithm will be utilized along with the weather cameras in Hawaii to provide as much information to pilots as possible," Colavito adds. One of the key advantages of VEIA is that it requires no specialized sensors to do its job, just the image feed from the web cams. Matthews recently accepted an R&D 100 Award for the algorithm, named one of the world's 100 most innovative products developed in 2019. As a researcher in air traffic management for 28 years, he is thrilled to have achieved this honor. "Some mountain passes in Alaska are like highways, especially in the summertime, with the number of people flying. You can find countless stories of terrible crashes, people just doing everyday things — a family on their way to a volleyball game," Matthews reflects. "I hope that VEIA might help people go about their lives safer." DISTRIBUTION STATEMENT A: Approved for public release. Distribution is unlimited. This material is based upon work supported by the Federal Aviation Administration under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Federal Aviation Administration. (Comment on this)

<< Previous Day

2020/01/06 [Calendar]

Next Day >>