Bigdl tutorial
Zhichao Li(zhichao.li@intel.com) Qiu xin([email protected]) Big Data Technologies, Software and Service Group, Intel
Intel® Confidential — INTERNAL USE ONLY About us
Software engineer at Intel, and the developer of BigDL Focusing area . Large scale machine learning, deep learning implementation and optimization . Machine learning / deep learning applications on big data
2 Agenda
Run things locally - Spark basic – Run BigDL(Keras model support) – Customer case Run in distributed way (Baiduyun) - LeNet - ResNet - Wide&deep
3 After this training, you can
. Know how to install and run BigDL . Get familiar with BigDL APIs . Build deep learning models (LeNet, Wide&deep and ResNet)on Apache Spark with BigDL and also Keras
4 Spark basic
5 The Big Data Problem
. One machine can not process or even store all the data ! . Solution is to distribute data over cluster of machine
6 7 Apache Spark
Apache Spark is a fast and general engine for large-scale data processing. • Up to 100x faster than Hadoop MapReduce in memory, or 10x faster on disk. • Unified engine/interface for complete data applications • SQL, Streaming, ML, Graph in the same framework • Write applications quickly in Java, Scala, Python, R • Runs on Hadoop, Mesos, standalone, or in the cloud (K8S is WIP) • Access diverse data sources including HDFS, Cassandra, HBase, and S3.
8 Apache Spark Components
SQL Streaming MLlib GraphX
Spark Core
9 How does Apache Spark work
Dataset (Memory, HDFS, S3, Servers Database)
Task ……
…… (Python, Scala…)
10 11 Hands on
Source code: - https://github.com/zhichao-li/act Machines: -
12 Bigdl introduction Why BigDL
13 Why BigDL?
Big data boost deep learning Production ML/DL system is Complex
Andrew NG, Baidu, NIPS 2015 Paper
14 Practical challenges
• Large scale dataset • Elasticity • Fault tolerance • Performance and scalability • Dynamic resource sharing • Integration with big data ecosystem • Programming tools / languages • Productivity …
15 Hadoop/Spark Ecosystem - Productivity
16 Apache Spark and BigDL - Productivity
SQL Streaming MLlib GraphX BigDL
Spark Core
17 Productivity
End-to-end machine learning pipeline • Large scale data management • Data analytics • Data cleaning and preprocess • Feature engineering • Hyper parameter tuning Fintech: Transaction fraud detection (Powered by Apache Spark and BigDL) … 18 Ease of Use
. A friendly API for model define, train, evaluate and predict . Support Scala and Python . Out of box run on Apache Spark . Easy integrate with other Apache Spark component . Scalable development . Easy to deployment . Visualization
19 Rich deep learning features
• Tensor, Layers – More than 100 (Linear, Conv2D, Conv3D, Embedding, Recurrent…) • Loss function – Dozens of loss functions(Cross Entropy, SmoothL1, DiceCoffient…) • Optimization algorithm – SGD, Adagrad, Adam… • Save and Load model files – Include torch / caffe / tensorflow
20 High performance from your server
. Powered by Intel Math Kernel Library . Extremely high performance on Xeon CPUs – Order of magnitude faster than out of box caffe / torch / tensorflow . Good scalability – Hundreds of nodes – https://www.cray.com/blog/scalable-deep-learning-bigdl-urika-xc- software-suite/
21 How BigDL run on Apache Spark
spark-submit
bigdl.jar your bigdl- python python MKL file native lib .zip
22 Bigdl introduction How to get BigDL
23 Get BigDL packages
. Pip install – Recommend for python user (only support BigDL on spark 2.2) . Download – If your spark is other version . Maven / Sbt – For Java/scala user . Build from source code – For BigDL developer Pip install $ pip install BigDL
25 Download https://bigdl-project.github.io/0.5.0/#release-download/
26 Maven / SBT
27 Build from source code
$ git clone https://github.com/intel-analytics/BigDL.git $ cd BigDL
$ ./make-dist.sh # For Spark 1.5/1.6 $ ./make-dist.sh -P spark_2.x # For Spark 2.0/2.1/2.2
28 Run BigDL program (pip install)
from bigdl.util.common import * from pyspark import SparkContext from bigdl.nn.layer import *
# create sparkcontext with bigdl configuration sc = SparkContext.getOrCreate(conf=create_spark_conf().setMaster("local[*]")) init_engine() # prepare the bigdl environment linear = Linear(2, 3) # Try to create a Linear layer
$ python your_python_file.py
29 Run BigDL program (on the cluster)
spark-submit \ --master xxx --jars path_to_big_dl_jar --py-files path_to_big_dl_python_zip your_python_file ……
30 Bigdl introduction Define, train and evaluate models
31 Define A Model
. Sequential API – In sequential API, user adds layers into some containers to build the model . Functional API – In functional API, the model is described as a graph
32 Lenet5
33 Define Lenet5 in Sequential API
model = Sequential() model.add(Reshape([1, 28, 28])) model.add(SpatialConvolution(1, 6, 5, 5)) model.add(Tanh()) model.add(SpatialMaxPooling(2, 2, 2, 2)) model.add(Tanh()) model.add(SpatialConvolution(6, 12, 5, 5)) model.add(SpatialMaxPooling(2, 2, 2, 2)) model.add(Reshape([12 * 4 * 4])) model.add(Linear(12 * 4 * 4, 100)) model.add(Tanh()) model.add(Linear(100, 10)) model.add(LogSoftMax())
34 Define Lenet5 in Functional API reshape1 = Reshape([1, 28, 28])() conv1 = SpatialConvolution(1, 6, 5, 5)(reshape1) tanh1 = Tanh()(conv1) pool1 = SpatialMaxPooling(2, 2, 2, 2)(tanh1) tanh2 = Tanh()(pool1) conv2 = SpatialConvolution(6, 12, 5, 5)(tanh2) pool2 = SpatialMaxPooling(2, 2, 2, 2)(conv2) reshape2 = Reshape([12 * 4 * 4])(pool2) linear1 = Linear(12 * 4 * 4, 100)(reshape2) tanh3 = Tanh()(linear1) linear2 = Linear(100, 10)(tanh3) softmax = LogSoftMax()(linear2)
model = Model(reshape1, softmax)
35 Keras Support
- Keras 1.2.2 Keras model Layers API - Load Keras Model - Keras-like API TensorFlow saved model Ops API
TensorFlow
36 Load Keras model
37 Keras-like API
38 Caffe Support
Load caffe model
model = Model.load_caffe_model(caffe.prototxt, caffe.model)
Load Caffe Model Weights to Predefined BigDL Model
model = Model.load_caffe(bigdlModel, caffe.prototxt, caffe.model, match_all=True)
39 Environment Setup
Before start this course, you should • Find a Linux/Mac machine
• Install python 2.7 and JDK 8 (the versions here are required by this course. BigDL also support python 3.5 and JDK7) • Run
pip install numpy scipy pandas matplotlib jupyter BigDL
export JAVA_HOME=jdk_path
jupyter notebook --notebook-dir=./ --ip=* --no-browser
40 Build digital number classifier with different models
Train different neural network models on MNIST dataset. . Introduce MNIST dataset . Logistic regression . CNN model
41 Notebook https://github.com/zhichao- li/act/blob/master/notebooks/part1/introduction_to_mnist.ipynb https://github.com/zhichao-li/act/blob/master/notebooks/part1/cnn.ipynb
42 Continue…
43 参与大会现场 互动赢取礼品
欢迎参观英特尔的展览 展位号：100 Train A Model
Optimizer . Model . Data – RDD[Sample] – Sample is actually an array of numpy ndarrays . Loss function . Batch size
45 Pipeline
Train(python RDD[raw data] Transform (python) RDD[Sample(ndarray,ndarray)] model)
46 Train A Model from bigdl.nn.layer import Linear from bigdl.util.common import * from bigdl.nn.criterion import MSECriterion from bigdl.optim.optimizer import Optimizer, MaxIteration import numpy as np model = Linear(2, 1) Define a model samples = [ Sample.from_ndarray(np.array([5, 5]), np.array([2.0])), Sample.from_ndarray(np.array([-5, -5]), np.array([-2.0])), Sample.from_ndarray(np.array([-2, 5]), np.array([1.3])), Produce some data Sample.from_ndarray(np.array([-5, 2]), np.array([0.1])), Sample.from_ndarray(np.array([5, -2]), np.array([-0.1])), Sample.from_ndarray(np.array([2, -5]), np.array([-1.3])) ] train_data = sc.parallelize(samples, 1) init_engine() optimizer = Optimizer(model, train_data, MSECriterion(), MaxIteration(100), 4) Define training process trained_model = optimizer.optimize()
47 Define when to end the training
optimizer = Optimizer(model, train_data, MSECriterion(), MaxEpoch(100), 4)
48 Change the optimization algorithm
optimizer = Optimizer(model, train_data, MSECriterion(), MaxIteration(100), 4, optim_method = Adam())
49 Validate your model in training optimizer.set_validation(batch_size, val_rdd, trigger, validationMethod)
• trigger: how often to do validation, maybe each several iterations or epochs • test data: the separate dataset for test • validation method: how to evaluate the model, maybe top1 accuracy, etc. • batch size: how many data evaluate in one time
50 Checkpointing optimizer.set_checkpoint(path, trigger,isOverWrite=True)
• path - the directory to save the snapshots • trigger - how often to save the check point
51 Visualization optimizer = Optimizer(...) ... log_dir = 'mylogdir' app_name = 'myapp' train_summary = TrainSummary(log_dir=log_dir, app_name=app_name) val_summary = ValidationSummary(log_dir=log_dir, app_name=app_name) optimizer.set_train_summary(train_summary) optimizer.set_val_summary(val_summary) ... trainedModel = optimizer.optimize()
52 Model Evaluation from bigdl.nn.layer import * from bigdl.util.common import * from bigdl.optim.optimizer import * import numpy as np sc = SparkContext.getOrCreate(conf=create_spark_conf()) init_engine() samples=[Sample.from_ndarray(np.array([1.0, 2.0]), np.array([2.0]))] testSet = sc.parallelize(samples,1)
//You can train a model or load an existing model before evaluation. model = Linear(2, 1) evaluateResult = model.evaluate(testSet, 1, [Top1Accuracy()]) print(evaluateResult[0])
53 Model Prediction
from bigdl.nn.layer import * from bigdl.util.common import * from bigdl.optim.optimizer import * import numpy as np
samples=[Sample.from_ndarray(np.array([.0, 2.0)]
predictSet = sc.parallelize(samples)
//train a model or load an existing model... //model = ... predictResult = model.predict(predictSet)
54 Distributed Evaluation and Prediction
Dataset (Memory, HDFS, S3, Servers
Database) ……
…… Model
55 BigDL model on Spark ML pipeline
What is Spark ML pipeline
56 BigDL model on Spark ML pipeline
What is Spark ML pipeline
57 BigDL model on Spark ML pipeline
Define a DLEstimator
linear_model = Sequential().add(Linear(2, 2)) mse_criterion = MSECriterion() estimator = DLEstimator( model=linear_model, criterion=mse_criterion, feature_size=[2], label_size=[2] )
58 BigDL model on Spark ML pipeline
Define a DLClassifier
linear_model = Sequential().add(Linear(2, 2)) classNLL_criterion = ClassNLLCriterion() Classifer = DLClassifier( model=linear_model, criterion=classNLL_criterion, feature_size=[2])
59 BigDL model on Spark ML pipeline
Hyperparameter setting
# for esitmator estimator.setBatchSize(4).setMaxEpoch(10).setLearningR ate(0.01) # for classifier classifier.setBatchSize(4).setMaxEpoch(10).setLearning Rate(0.01)
60 BigDL model on Spark ML pipeline
Training and prediction
# get a DLModel dlModel = estimator.fit(df) # get a DLClassifierModel dlClassifierModel = classifier.fit(df)
dlModel.transform(test_df).show(false)
61 Model Quantization
Quantize the model to get higher speed model = … quantizedModel = model.quantize()
62 Hands on
AutoEncoder Visualization
63 Build a distributed image prediction pipeline on Spark using BigDL
Model Pre-Process Prediction
Save Post-Process
https://github.com/intel-analytics/BigDL 64 Read from Distributed Cluster
Read Folder
Read Apache Parquet ImageFrame
RDD
https://github.com/intel-analytics/BigDL 65 Vision Library on top of OpenCV
Brightness ChannelOrder Hue Saturation Contrast ColorJitter Resize Hflip Expand Crop Normalize ……
Pascal VOC data sets (http://host.robots.ox.ac.uk/pascal/VOC/) 66 Data Loading and Pre-processing Pipeline
Read Folder ImageFrame Read Parquet BytesToMat RDD
…… MatToFloats Resize
https://github.com/intel-analytics/BigDL 67 Model Zoo Image Classification Object Detection • Inception • SSD (Single Shot • Resnet Multibox Detector) • VGG • VGG • MobileNet • MobileNet • Faster-RCNN • Alexnet • VGG • DenseNet • PvaNet • SqueezeNet https://github.com/intel-analytics/BigDL 68 Image Recognition and Object Detection
Pascal VOC data sets (http://host.robots.ox.ac.uk/pascal/VOC/) https://github.com/intel-analytics/BigDL Load model
https://github.com/intel-analytics/BigDL 70 Load model
https://github.com/intel-analytics/BigDL 71 Build a distributed image prediction pipeline on Spark using BigDL
Model Pre-Process Prediction
Save Post-Process https://github.com/intel-analytics/BigDL
72 Object Detection Video Demo
https://github.com/intel-analytics/analytics- zoo/blob/master/apps/ssd/messi.ipynb https://github.com/intel-analytics/BigDL 73 @ Problem
Large-scale image feature extraction • Object detect (remove background, optional) • Feature extraction Application • Similar image search • Image Deduplication
75 Similar image search
query
Search --- result
query Search result ---
------
76 Image Deduplication
HDFS SPARK 1, sampling & modeling 2, partitioning based on the mode according to similarity 3, do step 1 and 2 recursively Feature data: until the size of leaf node is below the threshold 1, image_id bound 2, feature 4, density based partitioning vector
Algorithm library libcluster.so
77 Feature Extraction pipeline
https://arxiv.org/pdf/1512.02325.pdf
http://mp.weixin.qq.com/s/xUCkzbHK4K06-v5qUsaNQQ (Chinese) https://software.intel.com/en-us/articles/building-large-scale-image-feature-extraction-with-bigdl-at-jdcom (English) Challenges of Large-Scale Processing in GPU Solutions
• Deploy deep learning inference application in GPU or FPGA is very expensive • GPU: The total count GPUs are much less than CPUs in our case. The most of GPUs are used to train models. • FPGA: requires a long life cycle to compile, test and deploy deep model to FPGA.
• Very complex and error-prone in managing large-scale GPU solutions in non- cluster mode • E.g., resource management and allocation, data partitioning, task balance, fault tolerance, model deployment, etc.
• Low parallelism in GPU solutions in cluster mode • GPU cores: 1 executor-cores and at most 4 num-executors are permitted due to limit of cards in case of 4 per server when using Caffe lib. • GPU memory: is very expensive and size is very limited. http://mp.weixin.qq.com/s/xUCkzbHK4K06-v5qUsaNQQ (Chinese) https://software.intel.com/en-us/articles/building-large-scale-image-feature-extraction-with-bigdl-at-jdcom (English) Upgrade to BigDL + Xeon Solutions
• Reuse existing Hadoop/Spark clusters for deep learning with no changes
• Efficiently scale out on Spark with superior performance • 3.83x speed-up when running on ~24 Xeon servers vs. 20 K40 GPU cards
• Very easy to build the end-to-end pipeline in BigDL • Library of image transformation and augmentation based on OpenCV val preProcessor = BytesToMat() -> Resize(300, 300) -> … val transformed = preProcessor(dataRdd)
• Loading pre-trained model (Caffe / Torch / TensorFLow) val model = SSDCaffeLoader.loadCaffe( caffeDefPath, caffeModelPath) Pipeline Correctness
Almost same as Caffe GPU Element-wise error < 0.001%
81 3.83x Speed up compared to GPU solution
82 More Information
• BigDL document https://bigdl-project.github.io/master/ • Ask questions: [email protected] • Report bug or raise feature request https://github.com/intel-analytics/BigDL/issues • Source code: https://github.com/intel-analytics/BigDL
83 Continue…
84 Run things in distributed Way
85 Pattern
Model Parallelism Data Parallelism
Source: Dean J, Corrado G, Monga R, et al. Large scale distributed deep networks[C]//Advances in neural information processing systems. 2012: 1223-1231.
86 Communication Model
Task Task
Task Driver Task
Task Task All to one Parameter Server 11 10 35 35 14 Task 4 7 35 2 Task 35 35 Task 35 5 1 35 6 2 5 1 6 Task
All reduce (tree aggregation) All reduce
87 Bulk Synchronous Parallel (BSP)
worker1 1 2 3
worker2 1 2 3
worker3 1 2 3
worker4 1 2 3
88 Asynchronous Synchronous Parallel (ASP)
worker1 1 2
worker2 1 2 3 4 5
worker3 1 2 3
worker4 1 2 3 4
Source: Dean J, Corrado G, Monga R, et al. Large scale distributed deep networks[C]//Advances in neural information processing systems. 2012: 1223-1231.
89 Logistic Regression on Spark* with Mini-Batch SGD Training Set Partition 1 Worker “Canonical” implementation Sample Repeat { 1 Driver broadcasts W to each worker 2 1 2 Workers compute gradient for the next batch of B records from the training set 3 Partition 2 Each task (running on workers) samples records from its data partition ퟏ Worker Driver Each task computes local gradient 품 = − 풚 ∙ 풙 1 3 풕 −풘푻∙풙 풊 풊 Sample ퟏ+풆 풊 푻 4 Aggregates gradient 푮 = ( 풕=ퟏ 품풕)/푩 (possibly through tree aggregation) 2 4 Driver updates weight 푊 = 푊 − η ∙ (G + 퐶 ∙ 푊) 3 }
1 …
Partition n Worker Sample
3 2
*Other names and brands may be claimed as the property of others.
90 Network and Memory Bottlenecks Large-scale problem settings Training Set . Model size: 100s of millions ~ billions unique features Partition 1 – Weight (W) and gradient (G) are both double vector, one entry for each unique feature Worker Training samples Sample . Training data: billions ~ trillions training samples cached in worker 2 – Partitioned & cached across workers memory 1
3 Partition 2 Worker Driver 1 Sample Broadcast W Each task (>800MB) to each 2 4 computes G worker in each (>800MB) in each 3 iteration
iteration … 1
Partition n Worker Each task sends G Sample (>800MB) for aggregation in each 3 2 iteration
91 Leveraging Data Sparsity
Usually real-world data are extremely sparse
. For our click through logs (100s of million unique features), each sample has several 100s ~ 1000 non- zero elements Gradient ퟏ . Switching to sparse vector (품풕 = 푻 − 풚풊 ∙ 풙풊) ퟏ+풆−풘 ∙풙풊 Solution
. Cached using sparse format
. Only Calc & sync gradient with non-zero data
. Using float values (instead of double values)
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. 92 BigDL Features Distributed Deep learning applications on Apache Spark* • No changes to the existing Hadoop/Spark clusters needed
https://software.intel.com/bigdl 94 BigDL Training Deep Dive
Highlight Block Manager(Parameter Server) • Implement an P2P All Reduce 3 5 3 5 3 5 Algorithm on Apache Spark Gradient Parameter 4 Gradient 4 Parameter Gradient 4 Parameter • Spark block manager as 2 2 2 parameter server (handle Worker Worker Worker
different APIs of Spark 1.x/2.x) 1 1 1 • Compress float32 parameter to … float16 parameter Partition 1 Partition 2 Partition n Training Set
95 Hands on
Source code • https://github.com/zhichao-li/act
Password: Intel123! ssh –D 8234 [email protected] Set socks5 proxy: localhost 8234 Open Jupyter ip: http://192.xxx.xxx.xxx:8889
96 Hands on
97 Hands on sh start_notebook.sh
Open Jupyter ip: http://192.xxx.xxx.xxx:8889
98 Hands on
CNN – keras-like version
Keras ResNet
Wide & Deep
99 参与大会现场 互动赢取礼品
欢迎参观英特尔的展览 展位号：100
Legal Disclaimer
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm
Intel, Quark, VTune, Xeon, Cilk, Atom, Look Inside and the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright ©2015 Intel Corporation.
102 Risk Factors
The above statements and any others in this document that refer to plans and expectations for the first quarter, the year and the future are forward-looking statements that involve a number of risks and uncertainties. Words such as “anticipates,” “expects,” “intends,” “plans,” “believes,” “seeks,” “estimates,” “may,” “will,” “should” and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions also identify forward-looking statements. Many factors could affect Intel’s actual results, and variances from Intel’s current expectations regarding such factors could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be the important factors that could cause actual results to differ materially from the company’s expectations. Demand could be different from Intel's expectations due to factors including changes in business and economic conditions; customer acceptance of Intel’s and competitors’ products; supply constraints and other disruptions affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Uncertainty in global economic and financial conditions poses a risk that consumers and businesses may defer purchases in response to negative financial events, which could negatively affect product demand and other related matters. Intel operates in intensely competitive industries that are characterized by a high percentage of costs that are fixed or difficult to reduce in the short term and product demand that is highly variable and difficult to forecast. Revenue and the gross margin percentage are affected by the timing of Intel product introductions and the demand for and market acceptance of Intel's products; actions taken by Intel's competitors, including product offerings and introductions, marketing programs and pricing pressures and Intel’s response to such actions; and Intel’s ability to respond quickly to technological developments and to incorporate new features into its products. The gross margin percentage could vary significantly from expectations based on capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; changes in revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; start-up costs; excess or obsolete inventory; changes in unit costs; defects or disruptions in the supply of materials or resources; product manufacturing quality/yields; and impairments of long-lived assets, including manufacturing, assembly/test and intangible assets. Intel's results could be affected by adverse economic, social, political and physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Expenses, particularly certain marketing and compensation expenses, as well as restructuring and asset impairment charges, vary depending on the level of demand for Intel's products and the level of revenue and profits. Intel’s results could be affected by the timing of closing of acquisitions and divestitures. Intel's results could be affected by adverse effects associated with product defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust, disclosure and other issues, such as the litigation and regulatory matters described in Intel's SEC reports. An unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular business practices, impacting Intel’s ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. A detailed discussion of these and other factors that could affect Intel’s results is included in Intel’s SEC filings, including the company’s most recent reports on Form 10-Q, Form 10-K and earnings release.
103