Building better batteries, faster

To help combat climate change, many car manufacturers are racing to add more electric vehicles in their lineups. But to convince prospective buyers, manufacturers need to improve how far these cars can go on a single charge. One of their main challenges? Figuring out how to make extremely powerful but lightweight batteries.

Typically, however, it takes decades for scientists to thoroughly test new battery materials, says Pablo Leon, an MIT graduate student in materials science. To accelerate this process, Leon is developing a machine-learning tool for scientists to automate one of the most time-consuming, yet key, steps in evaluating battery materials.

With his tool in hand, Leon plans to help search for new materials to enable the development of powerful and lightweight batteries. Such batteries would not only improve the range of EVs, but they could also unlock potential in other high-power systems, such as solar energy systems that continuously deliver power, even at night.

From a young age, Leon knew he wanted to pursue a PhD, hoping to one day become a professor of engineering, like his father. Growing up in College Station, Texas, home to Texas A&M University, where his father worked, many of Leon’s friends also had parents who were professors or affiliated with the university. Meanwhile, his mom worked outside the university, as a family counselor in a neighboring city.

In college, Leon followed in his father’s and older brother’s footsteps to become a mechanical engineer, earning his bachelor’s degree at Texas A&M. There, he learned how to model the behaviors of mechanical systems, such as a metal spring’s stiffness. But he wanted to delve deeper, down to the level of atoms, to understand exactly where these behaviors come from.

So, when Leon applied to graduate school at MIT, he switched fields to materials science, hoping to satisfy his curiosity. But the transition to a different field was “a really hard process,” Leon says, as he rushed to catch up to his peers.

To help with the transition, Leon sought out a congenial research advisor and found one in Rafael Gómez-Bombarelli, an assistant professor in the Department of Materials Science and Engineering (DMSE). “Because he’s from Spain and my parents are Peruvian, there’s a cultural ease with the way we talk,” Leon says. According to Gómez-Bombarelli, sometimes the two of them even discuss research in Spanish — a “rare treat.” That connection has empowered Leon to freely brainstorm ideas or talk through concerns with his advisor, enabling him to make significant progress in his research.

Leveraging machine learning to research battery materials

Scientists investigating new battery materials generally use computer simulations to understand how different combinations of materials perform. These simulations act as virtual microscopes for batteries, zooming in to see how materials interact at an atomic level. With these details, scientists can understand why certain combinations do better, guiding their search for high-performing materials.

But building accurate computer simulations is extremely time-intensive, taking years and sometimes even decades. “You need to know how every atom interacts with every other atom in your system,” Leon says. To create a computer model of these interactions, scientists first make a rough guess at a model using complex quantum mechanics calculations. They then compare the model with results from real-life experiments, manually tweaking different parts of the model, including the distances between atoms and the strength of chemical bonds, until the simulation matches real life.

With well-studied battery materials, the simulation process is somewhat easier. Scientists can buy simulation software that includes pre-made models, Leon says, but these models often have errors and still require additional tweaking.

To build accurate computer models more quickly, Leon is developing a machine-learning-based tool that can efficiently guide the trial-and-error process. “The hope with our machine learning framework is to not have to rely on proprietary models or do any hand-tuning,” he says. Leon has verified that for well-studied materials, his tool is as accurate as the manual method for building models.

With this system, scientists will have a single, standardized approach for building accurate models in lieu of the patchwork of approaches currently in place, Leon says.

Leon’s tool comes at an opportune time, when many scientists are investigating a new paradigm of batteries: solid-state batteries. Compared to traditional batteries, which contain liquid electrolytes, solid-state batteries are safer, lighter, and easier to manufacture. But creating versions of these batteries that are powerful enough for EVs or renewable energy storage is challenging.

This is largely because in battery chemistry, ions dislike flowing through solids and instead prefer liquids, in which atoms are spaced further apart. Still, scientists believe that with the right combination of materials, solid-state batteries can provide enough electricity for high-power systems, such as EVs. 

Leon plans to use his machine-learning tool to help look for good solid-state battery materials more quickly. After he finds some powerful candidates in simulations, he’ll work with other scientists to test out the new materials in real-world experiments.

Helping students navigate graduate school

To get to where he is today, doing exciting and impactful research, Leon credits his community of family and mentors. Because of his upbringing, Leon knew early on which steps he would need to take to get into graduate school and work toward becoming a professor. And he appreciates the privilege of his position, even more so as a Peruvian American, given that many Latino students are less likely to have access to the same resources. “I understand the academic pipeline in a way that I think a lot of minority groups in academia don’t,” he says.

Now, Leon is helping prospective graduate students from underrepresented backgrounds navigate the pipeline through the DMSE Application Assistance Program. Each fall, he mentors applicants for the DMSE PhD program at MIT, providing feedback on their applications and resumes. The assistance program is student-run and separate from the admissions process.

Knowing firsthand how invaluable mentorship is from his relationship with his advisor, Leon is also heavily involved in mentoring junior PhD students in his department. This past year, he served as the academic chair on his department’s graduate student organization, the Graduate Materials Council. With MIT still experiencing disruptions from Covid-19, Leon noticed a problem with student cohesiveness. “I realized that traditional [informal] modes of communication across [incoming class] years had been cut off,” he says, making it harder for junior students to get advice from their senior peers. “They didn’t have any community to fall back on.”

To help fix this problem, Leon served as a go-to mentor for many junior students. He helped second-year PhD students prepare for their doctoral qualification exam, an often-stressful rite of passage. He also hosted seminars for first-year students to teach them how to make the most of their classes and help them acclimate to the department’s fast-paced classes. For fun, Leon organized an axe-throwing event to further facilitate student cameraderie.

Leon’s efforts were met with success. Now, “newer students are building back the community,” he says, “so I feel like I can take a step back” from being academic chair. He will instead continue mentoring junior students through other programs within the department. He also plans to extend his community-building efforts among faculty and students, facilitating opportunities for students to find good mentors and work on impactful research. With these efforts, Leon hopes to help others along the academic pipeline that he’s become familiar with, journeying together over their PhDs.

Read More

Conduct what-if analyses with Amazon Forecast, up to 80% faster than before

Now with Amazon Forecast, you can seamlessly conduct what-if analyses up to 80% faster to analyze and quantify the potential impact of business levers on your demand forecasts. Forecast is a service that uses machine learning (ML) to generate accurate demand forecasts, without requiring any ML experience. Simulating scenarios through what-if analyses is a powerful business tool to navigate through the uncertainty of future events by capturing possible outcomes from hypothetical scenarios. It’s a common practice to assess the impact of business decisions on revenue or profitability, quantify the risk associated with market trends, evaluate how to organize logistics and workforce to meet customer demand, and much more.

Conducting a what-if analysis for demand forecasting can be challenging because you first need accurate models to forecast demand and then a quick and easy way to reproduce the forecast across a range of scenarios. Until now, although Forecast provided accurate demand forecasts, conducting what-if analysis using Forecast could be cumbersome and time-consuming. For example, retail promotion planning is a common application of what-if analysis to identify the optimal price point for a product to maximize the revenue. Previously on Forecast, you had to prepare and import a new input file for each scenario you wanted to test. If you wanted to test three different price points, you first had to create three new input files by manually transforming the data offline and then importing each file into Forecast separately. In effect, you were doing the same set of tasks for each and every scenario. Additionally, to compare scenarios, you had to download the prediction from each scenario individually and then merge them offline.

With today’s launch, you can easily conduct what-if analysis up to 80% faster. We have made it easy to create new scenarios by removing the need for offline data manipulation and import for each scenario. Now, you can define a scenario by transforming your initial dataset through simple operations, such as multiplying the price for product A by 90% or decreasing the price for product B by $10. These transformations can also be combined with conditions to control the parameters that the scenario applies in (for example, reducing product A’s price in one location only). With this launch, you can define and run multiple scenarios of the same type of analysis (such as promotion analysis) or different types of analyses (such as promotion analysis in geographical region 1 and inventory planning in geographical region 2) simultaneously. Lastly, you no longer need to merge and compare results of scenarios offline. Now, you can view the forecast predictions across all scenarios in the same graph or bulk export the data for offline review.

Solution overview

The steps in this post demonstrate how to use what-if analysis on the AWS Management Console. To directly use Forecast APIs for what-if analysis, follow the notebook in our GitHub repo that provides an analogous demonstration.

Import your training data

To conduct a what-if analysis, you must import two CSV files representing the target time series data (showing the prediction target) and the related time series data (showing attributes that impact the target). Our example target time series file contains the product item ID, timestamp, demand, store ID, city, and region, and our related time series file contains the product item ID, store ID, timestamp, city, region, and price.

To import your data, complete the following steps:

  1. On the Forecast console, choose View dataset groups.
Figure 1: View dataset group on the Amazon Forecast home page

Figure 1: View dataset group on the Amazon Forecast home page

  1. Choose Create dataset group.
Figure 2: Creating a dataset group

Figure 2: Creating a dataset group

  1. For Dataset group name, enter a dataset name (for this post, my_company_consumer_sales_history).
  2. For Forecasting domain, choose a forecasting domain (for this post, Retail).
  3. Choose Next.
Figure 3: Provide a dataset name and select your forecasting domain

Figure 3: Provide a dataset name and select your forecasting domain

  1. On the Create target time series dataset page, provide the dataset name, frequency of your data, and data schema
  2. Provide the dataset import details.
  3. Choose Start.

The following screenshot shows the information for the target time series page filled out for our example.

Figure 4: Sample information filled out for the target time series data import page

Figure 4: Sample information filled out for the target time series data import page

You will be taken to the dashboard that you can use to track progress.

  1. To import the related time series file, on the dashboard, choose Import.
Figure 5: Dashboard that allows you to track progress

Figure 5: Dashboard that allows you to track progress

  1. On the Create related time series dataset page, provide the dataset name and data schema.
  2. Provide the dataset import details.
  3. Choose Start.

The following screenshot shows the information filled out for our example.

Figure 6: Sample information filled out for the related time series data import page

Figure 6: Sample information filled out for the related time series data import page

Train a predictor

Next, we train a predictor.

  1. On the dashboard, choose Train predictor.
Figure 7: Dashboard of completed dataset import step and button to train a predictor

Figure 7: Dashboard of completed dataset import step and button to train a predictor

  1. On the Train predictor page, enter a name for your predictor, how long in the future you want to forecast and at what frequency, and the number of quantiles you want to forecast for.
  2. Enable AutoPredictor – this is required to use what-if analysis.
  3. Choose Create.

The following screenshot shows the information filled out for our example.

Figure 8: Sample information filled out to train a predictor

Figure 8: Sample information filled out to train a predictor

Create a forecast

After our predictor is trained (this can take approximately 2.5 hours), we create a forecast. You will know that your predictor is trained when you see the View Predictors button on your dashboard.

  1. Choose Create a forecast on the Dashboard

Figure 9: Dashboard of completed train predictor step and button to create a forecast

  1. On the Create a forecast page, enter a forecast name, choose the predictor that you created, and specify the forecast quantiles (optional) and the items to generate a forecast for.
  2. Choose Start.
Figure 10: Sample information filled out to create a forecast

Figure 10: Sample information filled out to create a forecast

After you complete these steps, you have successfully created a forecast. This represents your baseline forecast scenario that you use to do what-if analyses on.

If you need more help creating your baseline forecasts, refer to Getting Started (Console). We now move to the next steps of conducting a what-if analysis.

Create a what-if analysis

At this point, we have created our baseline forecast and will start the walkthrough of how to conduct a what-if analysis. There are three stages to conducting a what-if analysis: setting up the analysis, creating the what-if forecast by defining what is changed in the scenario, and comparing the results.

  1. To set up your analysis, choose Explore what-if analysis on the dashboard.
Figure 11: Dashboard of complete create forecast step and button to start what-if analysis

Figure 11: Dashboard of complete create forecast step and button to start what-if analysis

  1. Choose Create.
Figure 12: Page to create a new what-if analysis

Figure 12: Page to create a new what-if analysis

  1. Enter a unique name and select the baseline forecast on the drop-down menu.
  2. Choose the items in your dataset you want to conduct a what-if analysis for. You have two options:
    1. Select all items is the default, which we choose in this post.
    2. If you want to pick specific items, choose Select items with a file and import a CSV file containing the unique identifier for the corresponding item and any associated dimension (such as region).
  3. Choose Create what-if analysis.
Figure 13: Option to specify items to conduct what-if analysis for and button to create the analysis

Figure 13: Option to specify items to conduct what-if analysis for and button to create the analysis

Create a what-if forecast

Next, we create a what-if forecast to define the scenario we want to analyze.

  1. Choose Create.

Figure 14: Creating a what-if forecast

  1. Enter a name of your scenario.

You can define your scenario through two options:

  • Use transformation functions – Use the transformation builder to transform the related time series data you imported. For this walkthrough, we evaluate how the demand an item in our dataset changes when the price is reduced by 10% and then by 30% when compared to the price in the baseline forecast.
  • Define the what-if forecast with a replacement dataset – Replace the related time series dataset you imported.
Figure 15: Options to define a scenario

Figure 15: Options to define a scenario

The transformation function builder provides the capability to transform the related time series data you imported earlier through simple operations to add, subtract, divide, and multiply features in your data (for example price) by a value you specify. For our example, we create a scenario where we reduce the price by 10%, and price is a feature in the dataset.

  1. For What-if forecast definition method, select Use transformation functions.
  2. Choose Multiply as our operator, price as our time series, and enter 0.9.
Figure 16: Using the transformation builder to reduce price by 10%

Figure 16: Using the transformation builder to reduce price by 10%

You can also add conditions to further refine your scenario. For example if your dataset contained store information organized by region, you could limit the price reduction scenario by region. You could define a scenario of a 10% price reduction that’s applicable to stores not in Region_1.

  1. Choose Add condition.
  2. Choose Not equals as the operation and enter Region_1.
Figure 17: Using the transformation builder to reduce price by 10% for stores that are not in region 1

Figure 17: Using the transformation builder to reduce price by 10% for stores that are not in region 1

Another option to modify your related time series is by importing a new dataset that already contains the data defining the scenario. For example, to define a scenario with 10% price reduction, we can upload a new dataset specifying the unique identifier for the items that are changing and the price change that is 10% lower. To do so, select Define the what-if forecast with a replacement dataset and import a CSV containing the price change.

Figure 18: Importing a replacement dataset to define a new scenario

Figure 18: Importing a replacement dataset to define a new scenario

  1. To complete the what-if forecast definition, choose Create.
Figure 19: Completing the what-if forecast creation

Figure 19: Completing the what-if forecast creation

Repeat the process to create another what-if forecast with a 30% price reduction.

Figure 20: Showing the completed run of the two what-if forecasts

After the what-if analysis has run for each what-if forecast, the status will change to active. This concludes the second stage, and you can move on to comparing the what-if forecasts.

Compare the forecasts

We can now compare the what-if forecasts for both our scenarios, comparing a 10% price reduction with a 30% price reduction.

  1. On the analysis insights page, navigate to the Compare what-if forecasts section.

Figure 21: Inputs required to compare what-if forecasts

  1. For item_id, enter the item to analyze.
  2. For What-if forecasts, choose the scenarios to compare (for this post, Scenario_1 and Scenario_2).
  3. Choose Compare what-if.
Figure 22: button to generate what-if forecast comparison graph

Figure 22: button to generate what-if forecast comparison graph

The following graph shows the resulting demand in both our scenarios.

Figure 23: What-if forecast comparison for scenario 1 and 2

Figure 23: What-if forecast comparison for scenario 1 and 2

By default, it showcases the P50 and the base case scenario. You can view all quantiles generated by selecting your preferred quantiles on the Choose forecasts drop-down menu.

Export your data

To export your data to CSV, complete the following steps:

  1. Choose Create export.

Figure 24: Creating a what-if forecast export

  1. Enter a name for your export file (for this post, my_scenario_export)
  2. Specify the scenarios to be exported by selecting the scenarios on the What-If Forecast drop-down menu. You can export multiple scenarios at once in a combined file.
  3. For Export location, specify the Amazon Simple Storage Service (Amazon S3) location.
  4. To begin the export, choose Create Export.
Figure 25: specifying the scenario information and export location for the bulk export

Figure 25: specifying the scenario information and export location for the bulk export

  1. To download the export, first navigate to S3 file path location from the AWS Management Console and the select the file and choose the download button. The export file will contain the timestamp, item ID, dimensions, and the forecasts for each quantile for all scenarios selected (including the base scenario).

Conclusion

Scenario analysis is a critical tool to help navigate through the uncertainties of business. It provides foresight and a mechanism to stress-test ideas, leaving businesses more resilient, better prepared, and in control of their future. Forecast now supports forecasting what-if scenario analyses. To conduct your scenario analysis, open the Forecast console and follow the steps outlined in this post, or refer to our GitHub notebook on how to access the functionality via API.

To learn more, refer to the CreateWhatIfAnalysis page in the developer guide.


About the authors

Brandon Nair is a Sr. Product Manager for Amazon Forecast. His professional interest lies in creating scalable machine learning services and applications. Outside of work he can be found exploring national parks, perfecting his golf swing or planning an adventure trip.

Akhil Raj Azhikodan is a Software Development Engineer working on Amazon Forecast. His interests are in designing and building reliable systems that solve complex customer problems. Outside of work, he enjoys learning about history, hiking and playing video games.

Conner Smith is a Software Development Engineer working on Amazon Forecast. He focuses on building secure, scalable distributed systems that provide value to customers. Outside of work he spends time reading fiction, playing guitar, and watching random YouTube videos.

Shannon Killingsworth is the UX Designer for Amazon Forecast. He has been improving the user experience in Forecast for two years by simplifying processes as well as adding new features in ways that make sense to our users. Outside of work he enjoys running, drawing, and reading.

Read More

UVQ: Measuring YouTube’s Perceptual Video Quality

Online video sharing platforms, like YouTube, need to understand perceptual video quality (i.e., a user’s subjective perception of video quality) in order to better optimize and improve user experience. Video quality assessment (VQA) attempts to build a bridge between video signals and perceptual quality by using objective mathematical models to approximate the subjective opinions of users. Traditional video quality metrics, like peak signal-to-noise ratio (PSNR) and Video Multi-Method Assessment Fusion (VMAF), are reference-based and focus on the relative difference between the target and reference videos. Such metrics, which work best on professionally generated content (e.g., movies), assume the reference video is of pristine quality and that one can induce the target video’s absolute quality from the relative difference.

However, the majority of the videos that are uploaded on YouTube are user-generated content (UGC), which bring new challenges due to their remarkably high variability in video content and original quality. Most UGC uploads are non-pristine and the same amount of relative difference could imply very different perceptual quality impacts. For example, people tend to be less sensitive to the distortions of poor quality uploads than of high quality uploads. Thus, reference-based quality scores become inaccurate and inconsistent when used for UGC cases. Additionally, despite the high volume of UGC, there are currently limited UGC video quality assessment (UGC-VQA) datasets with quality labels. Existing UGC-VQA datasets are either small in size (e.g., LIVE-Qualcomm has 208 samples captured from 54 unique scenes), compared with datasets with millions of samples for classification and recognition (e.g., ImageNet and YouTube-8M), or don’t have enough content variability (sampling without considering content information, like LIVE-VQC and KoNViD-1k).

In “Rich Features for Perceptual Quality Assessment of UGC Videos“, published at CVPR 2021, we describe how we attempt to solve the UGC quality assessment problem by building a Universal Video Quality (UVQ) model that resembles a subjective quality assessment. The UVQ model uses subnetworks to analyze UGC quality from high-level semantic information to low-level pixel distortions, and provides a reliable quality score with rationale (leveraging comprehensive and interpretable quality labels). Moreover, to advance UGC-VQA and compression research, we enhance the open-sourced YouTube-UGC dataset, which contains 1.5K representative UGC samples from millions of UGC videos (distributed under the Creative Commons license) on YouTube. The updated dataset contains ground-truth labels for both original videos and corresponding transcoded versions, enabling us to better understand the relationship between video content and its perceptual quality.

Subjective Video Quality Assessment
To understand perceptual video quality, we leverage an internal crowd-sourcing platform to collect mean opinion scores (MOS) with a scale of 1–5, where 1 is the lowest quality and 5 is the highest quality, for no-reference use cases. We collect ground-truth labels from the YouTube-UGC dataset and categorize UGC factors that affect quality perception into three high-level categories: (1) content, (2) distortions, and (3) compression. For example, a video with no meaningful content won’t receive a high quality MOS. Also, distortions introduced during the video production phase and video compression artifacts introduced by third-party platforms, e.g., transcoding or transmission, will degrade the overall quality.

MOS= 2.052 MOS= 4.457
Left: A video with no meaningful content won’t receive a high quality MOS. Right: A video displaying intense sports shows a higher MOS.
MOS= 1.242 MOS= 4.522
Left: A blurry gaming video gets a very low quality MOS. Right: A video with professional rendering (high contrast and sharp edges, usually introduced in the video production phase) shows a high quality MOS.
MOS= 2.372 MOS= 4.646
Left: A heavily compressed video receives a low quality MOS. Right: a video without compression artifacts shows a high quality MOS.

We demonstrate that the left gaming video in the second row of the figure above has the lowest MOS (1.2), even lower than the video with no meaningful content. A possible explanation is that viewers may have higher video quality expectations for videos that have a clear narrative structure, like gaming videos, and the blur artifacts significantly reduce the perceptual quality of the video.

UVQ Model Framework
A common method for evaluating video quality is to design sophisticated features, and then map these features to a MOS. However, designing useful handcrafted features is difficult and time-consuming, even for domain experts. Also, the most useful existing handcrafted features were summarized from limited samples, which may not perform well on broader UGC cases. In contrast, machine learning is becoming more prominent in UGC-VQA because it can automatically learn features from large-scale samples.

A straightforward approach is to train a model from scratch on existing UGC quality datasets. However, this may not be feasible as there are limited quality UGC datasets. To overcome this limitation, we apply a self-supervised learning step to the UVQ model during training. This self-supervised step enables us to learn comprehensive quality-related features, without ground-truth MOS, from millions of raw videos.

Following the quality-related categories summarized from the subjective VQA, we develop the UVQ model with four novel subnetworks. The first three subnetworks, which we call ContentNet, DistortionNet and CompressionNet, are used to extract quality features (i.e., content, distortion and compression), and the fourth subnetwork, called AggregationNet, maps the extracted features to generate a single quality score. ContentNet is trained in a supervised learning fashion with UGC-specific content labels that are generated by the YouTube-8M model. DistortionNet is trained to detect common distortions, e.g., Gaussian blur and white noise of the original frame. CompressionNet focuses on video compression artifacts, whose training data are videos compressed with different bitrates. CompressionNet is trained using two compressed variants of the same content that are fed into the model to predict corresponding compression levels (with a higher score for more noticeable compression artifacts), with the implicit assumption that the higher bitrate version has a lower compression level.

The ContentNet, DistortionNet and CompressionNet subnetworks are trained on large-scale samples without ground-truth quality scores. Since video resolution is also an important quality factor, the resolution-sensitive subnetworks (CompressionNet and DistortionNet) are patch-based (i.e., each input frame is divided into multiple disjointed patches that are processed separately), which makes it possible to capture all detail on native resolution without downscaling. The three subnetworks extract quality features that are then concatenated by the fourth subnetwork, AggregationNet, to predict quality scores with domain ground-truth MOS from YouTube-UGC.

The UVQ training framework.

Analyzing Video Quality with UVQ
After building the UVQ model, we use it to analyze the video quality of samples pulled from YouTube-UGC and demonstrate that its subnetworks can provide a single quality score along with high-level quality indicators that can help us understand quality issues. For example, DistortionNet detects multiple visual artifacts, e.g., jitter and lens blur, for the middle video below, and CompressionNet detects that the bottom video has been heavily compressed.

ContentNet assigns content labels with corresponding probabilities in parentheses, i.e., car (0.58), vehicle (0.42), sports car (0.32), motorsports (0.18), racing (0.11).
DistortionNet detects and categorizes multiple visual distortions with corresponding probabilities in parentheses, i.e., jitter (0.112), color quantization (0.111), lens blur (0.108), denoise (0.107).
CompressionNet detects a high compression level of 0.892 for the video above.

Additionally, UVQ can provide patch-based feedback to locate quality issues. Below, UVQ reports that the quality of the first patch (patch at time t = 1) is good with a low compression level. However, the model identifies heavy compression artifacts in the next patch (patch at time t = 2).

Patch at time t = 1 Patch at time t = 2
Compression level = 0.000 Compression level = 0.904
UVQ detects a sudden quality degradation (high compression level) for a local patch.

In practice, UVQ can generate a video diagnostic report that includes a content description (e.g., strategy video game), distortion analysis (e.g., the video is blurry or pixelated) and compression level (e.g., low or high compression). Below, UVQ reports that the content quality, looking at individual features, is good, but the compression and distortion quality is low. When combining all three features, the overall quality is medium-low. We see that these findings are close to the rationale summarized by internal user experts, demonstrating that UVQ can reason through quality assessments, while providing a single quality score.

UVQ diagnostic report. ContentNet (CT): Video game, strategy video game, World of Warcraft, etc. DistortionNet (DT): multiplicative noise, Gaussian blur, color saturation, pixelate, etc. CompressionNet (CP): 0.559 (medium-high compression). Predicted quality score in [1, 5]: (CT, DT, CP) = (3.901, 3.216, 3.151), (CT+DT+CP) = 3.149 (medium-low quality).

Conclusion
We present the UVQ model, which generates a report with quality scores and insights that can be used to interpret UGC video perceptual quality. UVQ learns comprehensive quality related features from millions of UGC videos and provides a consistent view of quality interpretation for both no-reference and reference cases. To learn more, read our paper or visit our website to see YT-UGC videos and their subjective quality data. We also hope that the enhanced YouTube-UGC dataset enables more research in this space.

Acknowledgements
This work was possible through a collaboration spanning several Google teams. Key contributors include: Balu Adsumilli, Neil Birkbeck, Joong Gon Yim from YouTube and Junjie Ke, Hossein Talebi, Peyman Milanfar from Google Research. Thanks to Ross Wolf, Jayaprasanna Jayaraman, Carena Church, and Jessie Lin for their contributions.

Read More

Learn How Leading Companies Are Building AI Centers of Excellence, at NVIDIA GTC

AI Centers of Excellence are organizational units dedicated to implementing a company-wide AI vision. They help identify business use cases, create an implementation roadmap, accelerate adoption, assess impact and more.

NVIDIA GTC, a global conference on AI and the metaverse, brings together the world’s top business and technology leaders who’ve embraced artificial intelligence to transform their organizations.

The virtual conference, running Sept. 19-22, will feature talks from visionary leaders at companies including ByteDance, Deutsche Bank and Johnson & Johnson. Attend to explore real-world AI use cases, discover implementation strategies and get business tips from subject-matter experts across industries.

Register free for GTC and view the agenda for sessions on building AI Centers of Excellence.

Accelerating AI Adoption in Businesses

About 86% of business and tech executives expect AI to become a mainstream technology in their companies, according to a PwC survey. Such results show that advanced data analytics and AI software will be necessary for businesses to remain competitive across industries.

Industry leaders who take a holistic approach to data science and AI have experienced substantially greater benefits from AI initiatives compared with those who take a piecemeal approach. Reported advantages include about a 40% improvement in decision making, productivity through automation and customer experience, as well as the creation of more innovative products and services.

Accelerated computing platforms and frameworks now allow AI to be deployed quickly and at scale.

Check out the following GTC sessions for an inside look at how executives are driving AI adoption in the world’s most successful companies:

  • 5G Killer App: Making Augmented and Virtual Reality a Reality, featuring Bill Vass, vice president of engineering at AWS; Brian Mecum, vice president of device technology at Verizon; Peter Linder, head of 5G marketing for North America at Ericsson; and Veronica Yip, product manager and product marketing manager at NVIDIA.

Watch NVIDIA founder and CEO Jensen Huang’s GTC keynote on Tuesday, Sept. 20, at 8 a.m. PT, and browse the catalog of more than 250 sessions on using AI to build a future-ready business.

The post Learn How Leading Companies Are Building AI Centers of Excellence, at NVIDIA GTC appeared first on NVIDIA Blog.

Read More

Shelter From the Storm: AI Helps Gauge Catastrophe Risks

Floods in Kentucky and wildfires in California are the kinds of disasters companies of all sorts are trying to address with AI.

Tom Rikert, co-founder and CEO of San Francisco-based startup Masterful AI, is one of many experts helping them manage catastrophe risk.

In the U.S. alone, the National Association of Insurance Commissioners estimates that natural disasters cost $232 billion in 2019. In the face of such costs, regulators worldwide are pressing banks and corporations to be proactive.

“Institutions are expected to understand the impact of climate-related and environmental risks … in order to be able to make informed strategic and business decisions,” Europe’s central bank said in guidelines published in late 2020.

How AI Turns Data Into Insights

The good news is companies can access petabytes of geospatial images from daily satellite and drone feeds to detect and assess these risks.

“Humans can’t review all that data, so they need computer vision to find the patterns,” said Rikert. “That’s why AI’s now essential for managing catastrophe risk.”

Masterful AI uses semi-supervised machine learning, so companies only need to manually label a small fraction of the images they want to use to train AI models. It also supplies software to automate the job of quickly training models that sift through data for actionable insights.

More Flexible, Accurate Models

An analytics company recently used Masterful AI’s tools to detect and classify damaged or rusted transformers that could spark a wildfire, critical insights for utilities.

The software reduced error rates in AI models by over half and cut by two-thirds the time to build new models. Because Masterful can efficiently tap large volumes of unlabeled data for training, it also helped models detect more kinds of component defects across a wider range of background terrains.

“That’s a very high ROI for this field,” said Rikert, who earned a master’s degree from MIT in machine learning and an MBA from Harvard.

Growing Demand and Domains

The startup has worked with several customers and analysis firms that evaluate disasters, pollution and land-use plans. For example, it helped an insurance company improve damage assessments after disasters like hurricanes and hailstorms.

“We see a lot of demand from people who have data and models they’re trying to make more accurate and apply to more domains,” Rikert said.

Masterful AI builds and tests its models on PCs using a mix of NVIDIA GPUs, then runs its largest benchmarks on NVIDIA A100 Tensor Core GPUs in the cloud. Likewise, its customers use the tools locally and in the cloud.

From Laptops to the Cloud

“NVIDIA AI is very portable, so it’s easy to go from local development to a cloud deployment — that’s not the case for some platforms,” Rikert said.

Masterful AI also helps customers maximize their use of the memory packed in NVIDIA GPUs to accelerate training time.

“Without NVIDIA GPUs, we would not be able to accomplish our work,” he said. “It’s not feasible to train our models on CPUs, and we found NVIDIA GPUs have the best combination of operator support, performance and price compared to other accelerators.”

Synthetic Data Fills Gaps

Masterful AI is a member of NVIDIA Inception, a free, global program that helps startups access new technologies, expertise and investors. Thanks to Inception, Rikert’s team aims to test Omniverse Replicator to generate synthetic data that could further improve AI training.

Synthetic data is increasingly used to augment real-world datasets. It can improve AI performance on edge cases and situations where users lack real-world data.

“We see opportunities to improve AI model quality by helping optimize the mix of synthetic, labeled and unlabeled data customers use,” he said.

Broad Ecosystem for Risk Modeling

NVIDIA supports catastrophe-risk products from established software vendors and dozens of startups that are also members of Inception.

For example, Riskthinking.AI, in Toronto, uses probability models, augmented by AI, to create estimates of the financial impact of climate change. In addition, Heavy.ai in San Francisco provides GPU-accelerated analytics and visualization tools to help identify opportunities and risks hidden in massive geospatial and time-series datasets.

Lockheed Martin uses NVIDIA AI to help U.S. agencies fight wildfires. The UN Satellite Centre works with NVIDIA to manage climate risks and train data scientists on how to respond to floods.

Global solution integrators including Accenture, Deloitte and Ernst & Young also deliver NVIDIA-accelerated catastrophe risk products.

It’s a broad ecosystem fighting a growing set of disasters exacerbated by climate change.

“Catastrophes are unfortunately becoming more prevalent, so we’re using our experience working with customers and partners to help others get insights faster by automating their model development,” said Rikert.

Learn more about NVIDIA’s accelerated computing platform for financial services.

The post Shelter From the Storm: AI Helps Gauge Catastrophe Risks appeared first on NVIDIA Blog.

Read More

Predict, Detect, Mitigate: AI for Climate Science Takes the Stage at NVIDIA GTC

Recent AI advances enable modeling of weather forecasting 4-5 magnitudes faster than traditional computing methods.

The brightest leaders, researchers and developers in climate science, high performance computing and AI will discuss such technology breakthroughs — and how they can help foster a greener Earth — at NVIDIA GTC.

The virtual conference, running Sept. 19-22, also includes expert talks about more industries that will be transformed by AI, including healthcare, robotics, graphics and the industrial metaverse.

A dozen sessions will cover how accelerated computing can be used to predict, detect and mitigate climate-related issues. Some can’t-miss speakers include the following:

Register for free to attend GTC and discover how groundbreaking technologies are shaping the world. Add sessions focused on the clean energy transition to your conference agenda.

The post Predict, Detect, Mitigate: AI for Climate Science Takes the Stage at NVIDIA GTC appeared first on NVIDIA Blog.

Read More

3D Artists Reimagine, Remaster Iconic European Architecture This Week ‘In the NVIDIA Studio’

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology accelerates creative workflows. 

A triple threat steps In the NVIDIA Studio this week: a tantalizing trio of talented 3D artists who each reimagined and remastered classic European buildings with individualistic flair.

Robert Lazăr, Dawid Herda and Dalibor Cee have lived unique creative journeys — from their sources of inspiration; to the tricks they employ in their creative workflows; to the insights they’d share with up-and-coming artists.

NVIDIA Studio hardware and software powered the artists’ creative workflows.

While their techniques and styles may differ, they share an NVIDIA Studio-powered workflow. GPU acceleration in creative apps gave them the freedom to fast-track their artistry. AI-powered features accelerated by NVIDIA RTX GPUs reduced repetitive, tedious work, giving back valuable time for them to tinker with and perfect their projects.

Romanian Rendering 

Lazăr, who also goes by Eurosadboy, is a self-taught 3D artist with 17 years of experience, as well as an esteemed musician who embarks on a new adventure with each piece that he creates.

While exploring his hometown of Bucharest, Lazăr was delightfully overwhelmed by the Union of Romanian Architects building, with its striking fusion of nostalgia and futurism. Fueled by his passion for science fiction, he saw the opportunity to enhance this iconic building with digital art, featuring elements of the past, present and future.

Lazăr first surveyed the building on site to estimate general sizes, then created a moodboard to gather inspiration from his favorite artists.

An early iteration of Lazăr’s space lift structure in Cinema 4D.

“Considering that my style trends toward hyperrealism, and given the need for ray tracing in every scene, it was clear the GPU I chose had to be RTX,” Lazăr said.

With his vision in place, Lazăr opened Cinema 4D software and built models to bring the futuristic creation to life. The NVIDIA RTX GPU-accelerated viewport enabled smooth interactivity for these complex 3D shapes while modeling.

He then generated metal, stone and glass textures within the free JSplacement Classic software, then imported them back to Cinema 4D to apply them to his models. Animated elements were added to create his “space elevator” with rotating disks and unfolding arms.

To ensure the scene was lit identically to the original footage, Lazăr used GPU-accelerated ray tracing in Otoy’s Octane to create an ambient-occlusion effect, achieving photorealistic lighting with lightning speed.

Final compositing in Adobe After Effects accelerated by Lazăr’s GeForce RTX 3080 Laptop GPU.

At this stage, Lazăr imported the scene into Adobe After Effects software, then added the digital scene on top of the high-resolution video footage — creating an extraordinarily realistic visual. “The footage was in 4K RAW format, so without the capabilities of the NVIDIA RTX GPU, I wouldn’t have been able to preview in real time — making me spend more time on technical parts and less on creativity,” he said.

Matching colors was critical, the artist added, and thankfully After Effects’ several GPU-accelerated features, including Brightness & Contrast, Change Color and Exposure, helped him get the job done.

Making use of his GeForce 3080 Ti GPU and ASUS ProArt NVIDIA Studio laptop, Lazăr created this work of 3D art faster and more efficiently.

Polish Pride 

Dawid Herda, known widely as Graffit, has been an artist for more than a decade. He’s most inspired by his experiences hitchhiking across his home country, Poland.

Visiting Gdańsk, Herda found that the architecture of the city’s 600-year-old maritime crane sparked ideas for artistic transformation. He visualized the crane as a futuristic tower of metal and glass, drawing from the newer glass-fronted buildings that flank the old brick structure.

His workflow takes advantage of NVIDIA Omniverse, a platform for 3D design collaboration and world simulation, free for RTX GPU owners. The open-source, extensible Universal Scene Description file format gave Herda the freedom to work within several 3D apps at once, without having to repeatedly import and export between them. Plus, he shared his creation with fellow artists in real time, without his colleagues requiring advanced hardware.

“All these features make the job of complex design much more efficient, saving me a lot of time and freeing me to focus on creativity,” said Herda.

3D motion tracking in Blender.

Herda accessed the Omniverse Connector for Blender to accomplish 3D motion tracking, which is the simulation of live-action camera moves and perspective inside compositing software. From 4K ProRes footage of the crane captured by drone, Herda selected his favorite shots before importing them. He traced the camera movement and mapped perspective in the scene using specific points from the shots.

Blender with the AI denoising feature on vs. off.

“You often have to jump between apps, but thanks to NVIDIA Studio, everything becomes faster and smoother,” Herda said.

Then, Herda added his futuristic building variant, which was created and modeled from scratch. The AI denoising feature in the viewport and RTX GPU-accelerated ray tracing gave Herda instant feedback and crisp, beautiful details.

The artist made the foundational 3D model of the crane using simple blocks that were transformed by modeling and detailing each element. He swapped textures accurately in real time as he interacted with the model, achieving the futuristic look without having to wait for iterations of the model to render.

After animating each building shape, Herda quickly exported final frame renders using RTX-accelerated OptiX ray tracing. Then, he imported the project into After Effects, where GPU-accelerated features were used in the composite stage to round out the project.

His creative setup included a home PC equipped with a GeForce RTX 3090 GPU and an ASUS ZenBook Pro Duo NVIDIA Studio laptop with a GeForce RTX 3080 Laptop GPU. This meant Herda could create his photorealistic content anywhere, anytime.

Czech Craft 

Dalibor Cee turned a childhood fascination with 3D into a 20-year career. He started working with 3D architectural models before returning home to Prague to specialize in film special effects like fluid simulations, smoke and explosions.

Dalibor also enjoys projection mapping as a way to bring new light and feeling to old structures, such as the astronomical clock on the iconic Orloj building in Prague’s Old Town Square.

Fascinated by the circular elements of the clock, Dalibor reimagined them in his Czech sci-fi-inspired style by creating a lens effect and using shiny, golden elements and crystal shapes.

Dalibor applies various textures to multiple clock face layers.

The artist started in Blender for motion tracking to align his video footage with the 3D building blocks that would make up the main animation. Dalibor then added textures generated using the JSplacement tool. He experimented with colors, materials and masks to alter the glossiness or roughness, emission and specular aspects of each element.

Link objects on curves captured animations with keyframes in Blender.

“I use apps that need NVIDIA CUDA and PhysX, and generally all software has some advantage when used with NVIDIA RTX GPUs in 3D,” Dalibor said.

The models were then linked onto curves to be animated for forward, backward and rotating movements — similar to those of an optical zoom lens, creating animation depth. Dalibor achieved this with dramatic speed by using Blender Cycles RTX-accelerated OptiX ray tracing in the viewport.

This kind of work is very time and memory intensive, Dalibor said, but his two GeForce RTX 3090 Ti GPUs allow him to complete extra-large projects without having to waste hours on rendering. Blender’s Cycles engine with RTX-accelerated OptiX ray tracing and AI denoising enabled Dalibor to render the entire project in just 20 minutes — nearly 20x faster than with the CPU alone, according to his testing.

These time savings allowed Dalibor to focus on creating and animating the piece’s hundreds of elements. He combined colors and effects to bring the model to life in exactly the way he’d envisioned.

NVIDIA Studio systems have become essential for the next generation of 3D content creators, pushing boundaries to create inspirational, thought-provoking and emotionally intensive art.

Studio Success Stories

For a deeper understanding of their workflows, see how Lazăr, Herda and Dalibor brought their creations from concept to completion in their in-depth videos.

3D artists Lazăr, Herda and Dalibor.

Check out Lazăr’s and Herda’s Instagram channels and Dalibor on ArtStation.

Join the #CreatorsJourneyChallenge

In the spirit of learning, the NVIDIA Studio team is posing a challenge for the community to show off personal growth. Participate in the #CreatorsJourney challenge for a chance to be showcased on NVIDIA Studio social media channels.

Entering is easy. Post an older piece of artwork alongside a more recent one to showcase your growth as an artist. Follow and tag NVIDIA Studio on Instagram, Twitter or Facebook, and use the #CreatorsJourney tag to join.

Learn something new today: Access tutorials on the Studio YouTube channel and get creativity-inspiring updates directly to your inbox by subscribing to the NVIDIA Studio newsletter.

The post 3D Artists Reimagine, Remaster Iconic European Architecture This Week ‘In the NVIDIA Studio’ appeared first on NVIDIA Blog.

Read More

Intelligently search Alfresco content using Amazon Kendra

Amazon Kendra is an intelligent search service powered by machine learning (ML). With Amazon Kendra, you can easily aggregate content from a variety of content repositories into a centralized index that lets you quickly search all your enterprise data and find the most accurate answer. Many organizations use the content management platform Alfresco to store their content. One of the key requirements for many enterprise customers using Alfresco is the ability to easily and securely find accurate information across all the documents in the data source.

We are excited to announce the public preview of the Amazon Kendra Alfresco connector. You can index Alfresco content, filter the types of content you want to index, and easily search your data in Alfresco with Amazon Kendra intelligent search and its Alfresco OnPrem connector.

This post shows you how to use the Amazon Kendra Alfresco OnPrem connector to configure the connector as a data source for your Amazon Kendra index and search your Alfresco documents. Based on the configuration of the Alfresco connector, you can synchronize the connector to crawl and index different types of Alfresco content such as wikis and blogs. The connector also ingests the access control list (ACL) information for each file. The ACL information is used for user context filtering, where search results for a query are filtered by what a user has authorized access to.

Prerequisites

To try out the Amazon Kendra connector for Alfresco using this post as a reference, you need the following:

Configure the data source using the Amazon Kendra connector for Alfresco

To add a data source to your Amazon Kendra index using the Alfresco OnPrem connector, you can use an existing index or create a new index. Then complete the following steps. For more information on this topic, refer to the Amazon Kendra Developer Guide.

  1. On the Amazon Kendra console, open your index and choose Data sources in the navigation pane.
  2. Choose Add data source.
  3. Under Alfresco, choose Add connector.
  4. In the Specify data source details section, enter a name and description and choose Next.
  5. In the Define access and security section, for Alfresco site URL, enter the Alfresco host name.
  6. To configure the SSL certificates, you can create a self-signed certificate for this setup utilizing openssl x509 -in pattern.pem -out alfresco.crt and add this certificate to an Amazon Simple Storage Service (Amazon S3) bucket. Choose Browse S3 and choose the S3 bucket with the SSL certificate.
  7. For Site ID, enter the Alfresco site ID where you want to search documents.
  8. Under Authentication, you have two options:
    1. Use Secrets Manager to create new Alfresco authentication credentials. You need an Alfresco admin user name and password.
    2. Use an existing Secrets Manager secret that has the Alfresco authentication credentials you want the connector to access.
  9. Choose Save and add secret.
  10. For IAM role, choose Create a new role or choose an existing IAM role configured with appropriate IAM policies to access the Secrets Manager secret, Amazon Kendra index, and data source.
  11. Choose Next.
  12. In the Configure sync settings section, provide information about your sync scope and run schedule.
    You can include the files to be crawled using inclusion patterns or exclude them using exclusion patterns.
  13. Choose Next.
  14. In the Set field mappings section, you can optionally configure the field mappings to specify how the Alfresco field names are mapped to Amazon Kendra attributes or facets.
  15. Choose Next.
  16. Review your settings and confirm to add the data source.
  17. After the data source is added, choose Data sources in the navigation pane, select the newly added data source, and choose Sync now to start data source synchronization with the Amazon Kendra index.

    The sync process can take about 10–15 minutes. You can now search indexed Alfresco content using the search console or a search application. Optionally, you can search with ACL with the following additional steps.
  18. Go to the index page that you created and on the User access control tab, choose Edit settings.
  19. Under Access control settings, select Yes.
  20. For Token type, choose JSON.
  21. Choose Next.
  22. Choose Update.

Wait a few minutes for the index to get updated by the changes. Now let’s see how you can perform intelligent search with Amazon Kendra.

Perform intelligent search with Amazon Kendra

Before you try searching on the Amazon Kendra console or using the API, make sure that the data source sync is complete. To check, view the data sources and verify if the last sync was successful.

  1. To start your search, on the Amazon Kendra console, choose Search indexed content in the navigation pane.
    You’re redirected to the Amazon Kendra Search console. Now you can search information from the Alfresco documents you indexed using Amazon Kendra.
  2. For this post, we search for a document stored in Alfresco, AWS.
  3. Expand Test query with an access token and choose Apply token.
  4. For Username, enter the email address associated with your Alfresco account.
  5. Choose Apply.

Now the user can only see the content they have access to. In our example, user test@amazon.com doesn’t have access to any documents on Alfresco, so none are visible.

Limitations

The connector has the following limitations:

  • As of this writing, we only support Alfresco OnPrem. Alfresco PAAS is not supported.
  • The connector doesn’t crawl the following entities: calendars, discussions, data lists, links, and system files.
  • During public preview, we only support basic authentication. For support for other forms of authentication please contact your Amazon representative.

Clean up

To avoid incurring future costs, clean up the resources you created as part of this solution. If you created a new Amazon Kendra index while testing this solution, delete it. If you only added a new data source using the Amazon Kendra connector for Alfresco, delete that data source.

Conclusion

With the Amazon Kendra Alfresco connector, your organization can search contents securely using intelligent search powered by Amazon Kendra.

To learn more about the Amazon Kendra Alfresco connector, refer to the Amazon Kendra Developer Guide.

For more information on other Amazon Kendra built-in connectors to popular data sources, refer to Amazon Kendra native connectors.


About the author

Vikas Shah is an Enterprise Solutions Architect at Amazon web services. He is a technology enthusiast who enjoys helping customers find innovative solutions to complex business challenges. His areas of interest are ML, IoT, robotics and storage. In his spare time, Vikas enjoys building robots, hiking, and traveling.

Read More

CVNets: High Performance Library for Computer Vision

We introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation. CVNets supports image and video understanding tools, including data loading, data transformations, novel data sampling methods, and implementations of several standard networks with similar or better performance than previous studies.Apple Machine Learning Research