翻譯自：https://code.facebook.com/posts/366199913563917
快速有效的展示圖片對Facebook Android客戶端非常重要。可是該團隊多年來在有效存儲圖片時遇到了很多問題。圖片很大，可是設備卻很小。每個像素需要占用4字節的數據----red,green,blue和alpha值各占一字節。如果手機屏幕的尺寸是480*800的話，一張全屏的圖片會占用1.5M的內存。而手機的內存是有限的，Android設備給它的眾多應用程序分配各自的內存空間。在有些設備中，Facebook應用程序只被分配了16M的內存，這樣的情況下，一張圖片就幾乎占據了十分之一的內存。
當應用程序使用完內存會發生什麼情況呢？它會崩潰。因此，Facebook團隊創建了Fresco庫----它負責管理圖片及其使用的內存。
Android系統的內存區域
首先，我們需要了解Android系統提供的不同內存區域。
1）Java heap: 該區域被設備開發商對每個應用程序設定了嚴格的空間限制。所有使用Java語言的new操作符創建的對象都保存在這裡。此處的內存會被Java的垃圾回收機制處理，即當App不再使用該區域的某塊內存時，系統會自動回收它。
不幸的是，垃圾回收過程本身也是一個問題。當回收內存較多時，Android系統會完全停止應用程序以便執行回收。這也是應用程序凍屏或卡頓的最常見的應用。這些很影響用戶的體驗，他們此時可能正在滑動或點擊按鈕，卻只能等待應用程序做出反應。
2）Native heap： 使用C++語音的new操作符創建的對象放在這個區域。在該區域App只受設備的物理內存的限制。並且沒有垃圾回收機制，不用擔心應用程序卡頓的問題。但是，C++應用程序需要負責釋放分配的所有內存，否則會引起內存洩漏，最終導致應用程序崩潰。
3）ashmem:也叫匿名共享內存。它的行為與native heap很像，但是多了額外的系統調用。Android系統可以“unpin"該內存而不用釋放它。只是一種懶惰模式的釋放，內存只有在系統趨勢需要更多的內存時才真正被釋放。當Android系統"pin"該內存後，如果該內存沒有被釋放過的話，舊的數據仍然存在。
可清除的位圖（Purgeable bitmaps)
一般情況下，Java應用程序不能夠直接對Ashmem區域進行操作，但是有些情況例外，圖片就是其一。當插件一個解壓後的圖片（即未被壓縮的圖片）時，即Bitmap， Android API提供了purgeable選項：
BitmapFactory.Options = new BitmapFactory.Options();
options.inPurgeable = true;Bitmap bitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length, options);
設置該選項的圖片保存在ashmem內存區。因此，垃圾回收機制不會自動回收它們。Android的系統庫在繪制系統需要渲染該圖片時，對該圖片的內存區域執行"pin"操作，結束後執行”unpin". 被unpin過的內存會隨時被系統回收。當unpin過的圖片需要重繪時，系統會即時重新解壓該圖片。
這似乎是一個很完美的解決方案，但問題是即時的解壓發生在UI線程中。解壓是一個很耗費CUP的操作，會引起UI卡頓。基於上述原因，Google現在不再建議使用這一特性，而推薦使用另一個選項：inBitmap. 該選項的問題是Android3.0之後才出現。並且在Android 4.4之前只在圖片的大小相同時才起作用。Fresco就是解決這個問題的，該庫甚至可以在Android 2.3系統上使用。
Fresco的解決方法
找到了一種解決上述問題----快速的UI和快速的內存的方法。如果提前在非UI線程中pin內存，並保證不會被unpin的話,這樣既能把圖片保存在ashmem中又不會引起UI卡頓。Fresco團隊發現Android Native Development Kit (NDK)中的函數AndroidBitmap_lockPixels實現了pin功能，但是Android系統中使用該函數後會使用unlockPixels將內存unpin.Fresco的突破是調用lockPixels卻不使用配對的unlockPixels, 這樣創建的圖片保存在Java heap之外並且不會影響UI線程的速度。
用Java寫代碼時，像C++一樣考慮內存的釋放問題
使用pin操作過的可清除位圖既不會被垃圾回收又不享受ashmem內置的回收機制以防止內存洩漏。它完全依靠程序開發人員來維護。
在C++中，常用的解決方案是使用智能指針來實現引用計數。它利用了C++語言的各種機制---拷貝構造函數，assignment操作符，及析構函數。這些在Java語言中都不存在，因此我們需要找出一種方法使得使用Java語言也可以得到C++語言類似的保證。
Fresco使用了兩個類來實現上述方法。一個是SharedReference類， 該類有兩個方法：addReference和deleteReference, 調用者在使用被引用的對象或不再使用該對象時必須調用這兩個方法。當引用數為0時，被引用的對象被回收（如使用Bitmap.recycle).
很明顯，java開發人員很容易忘記調用上述方法，因為避免開發人員自己管理對象的釋放是java的一大特性。因此，在SharedReference類上，Fresco又創建了CloseableReference類，它不僅實現了java的Closeable接口，還實現了Cloneable接口。構造函數和clone()方法調用addReference()，而close()方法調用deleteReference()。 因此Java開發者只需要准從如下兩個簡單的規則：
1.在將一個CloseableReference對象賦值給新對象時，使用.clone()。
2.在退出對象的作用域之前，調用.close(), 通常在finally模塊中使用。
上述規則既有效的防止了內存洩漏，又讓我們在大型java應用程序中享受native內存管理。
Fresco不僅僅是個圖片加載器，而是管道
將圖片展示到移動設備需要如下步驟：
很多已知的開源庫也采用這樣的步驟，如Picasso,Universal Image Loader,Glide,Volley等。所有這些庫對Android開發做出了重要的貢獻，而Fresco在一些重要的方面走得更遠。
將這些步驟看作管道而不是加載器就是一個重要的不同點。每一個步驟都盡可能地與其他步驟獨立，每個步驟都接收一個輸入和一些參數，然後產出一個輸出。它使得並行做這些操作成為可能，而不是像其他庫那樣只能順序操作。有些操作只在特定的條件下執行，有些操作可以對執行的線程做一些特殊的要求。很多用戶在網上很慢的情況下使用Facebook，這時候我們希望用戶能盡可能快的看到圖片，甚至在圖片還沒有完全下載完之前。
停止擔心，享受流的概念
Java中的異步代碼通常通過類似Future的機制來執行。代碼被提交到另外一個線程執行，使用一個類似Future的對象去確認結果是否已經准備好了。該操作的一個缺陷是假設只有一個結果。當處理漸進性圖片時，我們需要一系列正在處理的結果。
Fresco的解決方案是使用Future的更同樣的版本，DataSource類。它提供了一個訂閱方法，在該方法中，調用者必須傳遞一個DataSubscriber對象和一個Executor對象。該DataSubscriber對象接收來自DataSource的所有中間結果及最後的結果，並提供一個簡單的方法區分它們。因為我們會頻繁使用需要顯示close調用的對象，因此DataSource本身是一個Closeable對象。
在後台，使用了新的叫做Producer/Consumer的框架來實現上述方案。它是從ReactiveX框架獲取的靈感，使用與RxJava類似的接口，但是更適合移動端並且含有內在的Closeables支持。
接口很簡單，Producer類只有一個方法produceResults，該方法使用Consumer對象為參數。對應地，Consumer類有一個onNewResult方法。
就像Java的輸入輸出流一樣，我們可以將多個生產者串聯起來做成一個生產鏈。假設我們有一個生產者，它的任務是將I類型轉換為O類型。可以通過類似下圖的方法來生成生產鏈：
public class OutputProducer implements Producer {
private final Producer mInputProducer;
public OutputProducer(Producer inputProducer) {
this.mInputProducer = inputProducer;
}
public void produceResults(Consumer outputConsumer, ProducerContext context) {
Consumer inputConsumer = new InputConsumer(outputConsumer);
mInputProducer.produceResults(inputConsumer, context);
}
private static class InputConsumer implements Consumer {
private final Consumer mOutputConsumer;
public InputConsumer(Consumer outputConsumer) {
mOutputConsumer = outputConsumer;
}
public void onNewResult(I newResult, boolean isLast) {
O output = doActualWork(newResult);
mOutputConsumer.onNewResult(output, isLast);
}
}}
這使得我們可以把一系列的步驟鏈接起來，而同時卻保持它們的邏輯獨立性。
動畫——從一到多
Stickers, 一種使用GIF和WebP格式保存的動畫，深受Facebook用戶的喜愛。在移動端支持該動畫會引起新的挑戰。動畫不僅是一個位圖而是一系列的位圖，每張都需要解碼，保存在內存中，然後展現。將每一幀動畫都保存在內存中對大型動畫來說是不現實的。
Fresco創建了AnimatedDrawable類，具備渲染動畫能力的圖片類，及兩個支持類---一個支持GIF，另一個支持WebP。AnimatedDrawable類實現了標准的AndroidAnimatable接口，因此調用者可以隨時啟動和終止動畫。為優化內存使用，只在所有幀都足夠小時才全部緩存，如果太大的話，就即時解碼。該行為對調用者是完全可行的。
兩個支持類是使用C++編碼實現的。我們只保存解碼後的數據和解析後的元數據（如寬度和高度等）的一份拷貝，並保存對數據的引用計數，以便在Java端允許多個Drawable對象同時讀取同一個WebP圖片。
Drawee——圖片的展示
當圖片正在從網絡上下載時，我們希望現實一個占位圖。如果下載失敗的話，顯示一個錯誤指示圖。當圖片下載完成後，我們做一個快速的fade-in動畫。
我們經常需要放大圖片，或甚至應用一個展示矩陣去使用硬件加速以固定尺寸來渲染圖片。我們並不是每次都以圖片中心點為焦點來進行放大，焦點可以在任何地方。有時我們需要圓角的圖片，甚至圓形的圖片。所有這些操作都需要快速平滑。
最開始的實現是使用Android的View對象——當圖片下載完成後，替換占位頁面為一個ImageView。該方案被證明速度非常慢。改變頁面會促使Android系統執行整個的布局過程（layout pass)，而這是用戶滑動時最不希望看到的。可行的方案是使用Android的圖片類（Drawable)，可以被快速的替換。
因此Fresco創建了Drawee，它是一個MVC風格的展示圖片的框架。使用DraweeHierarchy類來實現該框架，該類實現了圖片的分層制度，每層對應一個具體的功能——對原始圖片進行展示，分層，淡入或放大等功能。
DraweeControllers類連接圖片管道（或任何圖片加載器）並負責後台的圖片操作。他們從管道中接收事件並決定如何處理它們。他們控制DraweeHierachy的展示——是占位圖，還是錯誤指示圖，或者下載完成的圖片。
DraweeViews類只提供有些的功能，但它提供的功能卻是決定性的。它監聽Android系統中頁面是否還在顯示的事件。當頁面不再顯示時通知DraweeController關閉圖片使用的資源。這避免了內存洩漏。另外，上述controller會通知頁面管道取消網絡請求，如果該請求還沒有完成的話。因此，滑動很長的圖片列表，不會引起網絡阻塞。
上述機制解決了展示圖片的大部分工作。調用代碼只需要創建一個DraweeView對象，指定URI，並且選擇性的添加也許參數即可。開發者不需要擔心如何管理圖片內存或更新圖片的問題。所有這些事情都由Fresco庫做了處理。
原文如下，寫得非常好，提供給願意看原文的：
Displaying images quickly and efficiently on Facebook for Android is important. Yet we have had many problems storing images effectively over the years. Images are large, but devices are small. Each pixel takes up 4 bytes of data — one for each of red, green, blue, and alpha. If a phone has a screen size of 480 x 800 pixels, a single full-screen image will take up 1.5 MB of memory. Phones often have very little memory, and Android devices pide up what memory they have among multiple apps. On some devices, the Facebook app is given as little as 16 MB — and justoneimage could take up a tenth of that!
What happens when your app runs out of memory? It crashes. We set out to solve this by creating a library we're calling Fresco — it manages images and the memory they use. Crashes begone.
Regions of memory
To understand what Facebook did here, we need to understand the different heaps of memory available on Android.
TheJavaheapis the one subject to the strict, per-application limits set by the device manufacturer. All objects created using the Java language'snewoperator go here. This is a relatively safe area of memory to use. Memory isgarbage-collected, so when the app has finished with memory, the system will automatically reclaim it.
Unfortunately, this process of garbage collection is precisely the problem. To do more than basic reclamations of memory, Android must halt the application completely while it carries out the garbage collection. This is one of the most common causes of an app appearing to freeze or stall briefly while you are using it. It's frustrating for people using the app, and they may try to scroll or press a button — only to see the app wait inexplicably before responding.
In contrast, thenativeheapis the one used by the C++ new operator. There is much more memory available here. The app is limited only by the physical memory available on the device. There is no garbage collection and nothing to slow things down. However, C++ programs are responsible for freeing every byte of memory they allocate, or they will leak memory and eventually crash.
Android has another region of memory, calledashmem. This operates much like the native heap, but has additional system calls. Android can “unpin” the memory rather than freeing it. This is a lazy free; the memory is freed only if the system actually needs more memory. When Android “pins” the memory back, old data will still be there if it hasn't been freed.
Purgeable bitmaps
Ashmem is not directly accessible to Java applications, but there are a few exceptions, and images are one of them. When you create a decoded (uncompressed) image, known as abitmap, the Android API allows you to specify that the image bepurgeable:
BitmapFactory.Options = new BitmapFactory.Options();
options.inPurgeable = true;Bitmap bitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length, options);
Purgeable bitmaps live in ashmem. However, the garbage collector does not automatically reclaim them. Android's system libraries “pin” the memory when the draw system is rendering the image, and “unpin” it when it's finished. Memory that is unpinned can be reclaimed by the system at any time. If an unpinned image ever needs to be drawn again, the system will just decode it again, on the fly.
This might seem like a perfect solution, but the problem is that the on-the-fly decode happens on the UI thread. Decoding is a CPU-intensive operation, and the UI will stall while it is being carried out. For this reason, Google nowadvises against using the feature. They now recommend using a different flag,inBitmap. However, this flag did not exist until Android 3.0. Even then, it was not useful unless most of the images in the app were the same size, which definitely isn't the case for Facebook. It was not until Android 4.4 that this limitation was removed. However, we needed a solution that would work for everyone using Facebook, including those running Android 2.3.
Having our cake and eating it too
We found a solution that allows us to have the best of both worlds — both a fast UI and fast memory. If we pinned the memory in advance, off the UI thread, and made sure it was never unpinned, then we could keep the images in ashmem but not suffer the UI stalls. As luck would have it, the Android Native Development Kit (NDK) has afunction that does precisely this, calledAndroidBitmap_lockPixels. The function was originally intended to be followed by a call tounlockPixelsto unpin the memory again.
Our breakthrough came when we realized we didn't have to do that. If we called lockPixelswithouta matching unlockPixels, we created an image that lived safely off the Java heap and yet never slowed down the UI thread. A few lines of C++ code, and we were home free.
Write code in Java, but think like C++
As we learned from Spider-Man, “With great power comes great responsibility.” Pinned purgeable bitmaps have neither the garbage collector's nor ashmem's built-in purging facility to protect them from memory leaks. We are truly on our own.
In C++, the usual solution is to build smart pointer classes that implement reference counting. These make use of C++ language facilities — copy constructors, assignment operators, and deterministic destructors. This syntactic sugar does not exist in Java, where the garbage collector is assumed to be able to take care of everything. So we have to somehow find a way to implement C++-style guarantees in Java.
We made use of two classes to do this. One is simply calledSharedReference. This has two methods, addReference and deleteReference, which callers must call whenever they take the underlying object or let it out of scope. Once the reference count goes to zero, resource disposal (such asBitmap.recycle) takes place.
Yet, obviously, it would be highly error-prone to require Java developers to call these methods. Java was chosen as a language toavoiddoing this! So on top of SharedReference, we builtCloseableReference. This implements not only the JavaCloseableinterface, butCloneableas well. The constructor and theclone()method calladdReference(), and theclose()method callsdeleteReference(). So Java developers need only follow two simple rules:
- On assigning a CloseableReference to a new object, call
.clone().Before going out of scope, call.close(), usually in a finally block.
These rules have been effective in preventing memory leaks, and have let us enjoy native memory management in large Java applications like Facebook for Android and Messenger for Android.
It's more than a loader — it's apipeline
There are many steps involved in showing an image on a mobile device:
Several excellent open source libraries exist that perform these sequences —Picasso,Universal Image Loader,Glide, andVolley, to name a few. All of these have made important contributions to Android development. We believe our new library goes further in several important ways.
Thinking of the steps as apipelinerather than as aloaderin itself makes a difference. Each step should be as independent of the others as possible, taking an input and some parameters and producing an output. It should be possible to do some operations in parallel, others in serial. Some execute only in specific conditions. Several have particular requirements as to which threads they execute on. Moreover, the entire picture becomes more complex when we consider progressive images. Many people use Facebook over very slow Internet connections. We want these users to be able to see their images as quickly as possible, often even before the image has actually finished downloading.
Stop worrying, love streaming
Asynchronous code on Java has traditionally been executed through mechanisms likeFuture. Code is submitted for execution on another thread, and an object like a Future can be checked to see if the result is ready. This, however, assumes that there is only one result. When dealing with progressive images, we want there to be an entire series of continuous results.
Our solution was a more generalized version of Future, calledDataSource. This offers a subscribe method, to which callers must pass aDataSubscriberand anExecutor. The DataSubscriber receives notifications from the DataSource on both intermediate and final results, and offers a simple way to distinguish between them. Because we are so often dealing with objects that require an explicitclosecall, DataSource itself is aCloseable.
Behind the scenes, each of the boxes above is implemented using a new framework, called Producer/Consumer. Here we drew inspiration fromReactiveXframeworks. Our system has interfaces similar toRxJava, but more appropriate for mobile and with built-in support for Closeables.
The interfaces are kept simple.Producerhas a single method,produceResults, which takes aConsumerobject. Consumer, in turn, has anonNewResultmethod.
We use a system like this to chain producers together. Suppose we have a producer whose job is to transform type I to type O. It would look like this:
public class OutputProducer implements Producer {
private final Producer mInputProducer;
public OutputProducer(Producer inputProducer) {
this.mInputProducer = inputProducer;
}
public void produceResults(Consumer outputConsumer, ProducerContext context) {
Consumer inputConsumer = new InputConsumer(outputConsumer);
mInputProducer.produceResults(inputConsumer, context);
}
private static class InputConsumer implements Consumer {
private final Consumer mOutputConsumer;
public InputConsumer(Consumer outputConsumer) {
mOutputConsumer = outputConsumer;
}
public void onNewResult(I newResult, boolean isLast) {
O output = doActualWork(newResult);
mOutputConsumer.onNewResult(output, isLast);
}
}}
This lets us chain together a very complex series of steps and still keep them logically independent.
Animations — from one to many
Stickers, which are animations stored in the GIF and WebP formats, are well liked by people who use Facebook. Supporting them poses new challenges. An animation is not one bitmap but a whole series of them, each of which must be decoded, stored in memory, and displayed. Storing every single frame in memory is not tenable for large animations.
We builtAnimatedDrawable, a Drawable capable of rendering animations, and two backends for it — one for GIF, the other for WebP. AnimatedDrawable implements the standard AndroidAnimatableinterface, so callers can start and stop the animation whenever they want. To optimize memory usage, we cache all the frames in memory if they are small enough, but if they are too large for that, we decode on the fly. This behavior is fully tunable by the caller.
Both backends are implemented in C++ code. We keep a copy of both the encoded data and parsed metadata, such as width and height. We reference count the data, which allows multiple Drawables on the Java side to access a single WebP image simultaneously.
How do I love thee? Let me Drawee the ways . . .
When images are being downloaded from the network, we want to show a placeholder. If they fail to download, we show an error indicator. When the image does arrive, we do a quick fade-in animation. Often we scale the image, or even apply a display matrix, to render it at a desired size using hardware acceleration. And we don't always scale around the center of the image — the useful focus point may well be elsewhere. Sometimes we want to show the image with rounded corners, or even as a circle. All of these operations need to be fast and smooth.
Our previous implementation involved using AndroidViewobjects — swapping out a placeholder View for anImageViewwhen the time came. This turned out to be quite slow. Changing Views forces Android to execute an entire layout pass, definitely not something you want to happen while users are scrolling. A more sensible approach would be to use Android's Drawables, which can be swapped out on the fly.
So we built Drawee. This is an MVC-like framework for the display of images. The model is calledDraweeHierarchy. It is implemented as a hierarchy of Drawables, each of which applies a specific function — imaging, layering, fade-in, or scaling — to the underlying image.
DraweeControllersconnect to the image pipeline — or to any image loader — and take care of backend image manipulation. They receive events back from the pipeline and decide how to handle them. They control what the DraweeHierarchy actually displays — whether a placeholder, error condition, or finished image.
DraweeViewshave only limited functionality, but what they provide is decisive. They listen for Android system events that signal that the view is no longer being shown on-screen. When going off-screen, the DraweeView can tell the DraweeController to close the resources used by the image. This avoids memory leaks. In addition, the controller will tell the image pipeline to cancel the network request, if it hasn't gone out yet. Thus, scrolling through a long list of images, as Facebook often does, will not break the network bank.
With these facilities, the hard work of displaying images is gone. Calling code need only instantiate a DraweeView, specify a URI, and, optionally, name some other parameters. Everything else happens automatically. Developers don't need to worry about managing image memory or streaming the updates to the image. Everything is done for them by the libraries.
Fresco
Having built this elaborate tool set for image display and manipulation, we wanted to share it with the Android developer community. We are pleased to announce that, as of today, this project is now available asopen source.
Afrescois a painting technique that has been popular around the world for centuries. With this name, we honor the many great artists who have used this form, from Italian Renaissance masters like Raphael to the Sigiriya artists of Sri Lanka. We don't pretend to be on that level. We do hope that Android app developers enjoy using our library as much as we've enjoyed building it.
