OkHttp 离线缓存实现

发表于   |  

Okhttp 本身已经支持标准的 http 协议缓存策略了， 本文将进一步压榨OkHttp，让它在实现离线缓存。

场景应用:

OkHttp 内部已经支持了标准的 Http 协议缓存策略，如 Last-Modified, Etag, Cache-Control 等方式，看起来已经是非常够用了，但是我们还想让缓存在这样的场景下得到使用: 当客户端设备网络中断或者服务端出现了错误，但是本地存在缓存副本的时候，我们依然想取用这部分缓存。

OkHttp 保存缓存的时机:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
okhttp3.internal.http.HttpEngine:
private void maybeCache() throws IOException {
    InternalCache responseCache = Internal.instance.internalCache(client);
    if (responseCache == null) return;

    // Should we cache this response for this request?
    if (!CacheStrategy.isCacheable(userResponse, networkRequest)) {
      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          responseCache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.
        }
      }
      return;
    }

    // Offer this request to the cache.
    storeRequest = responseCache.put(stripBody(userResponse));
}

这里我们可以看到 OkHttp 是否缓存请求结果其中的一个条件:CacheStrategy.isCacheable(), 看看这里面的代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
okhttp3.internal.http.CacheStrategy:
public static boolean isCacheable(Response response, Request request) {
    // Always go to network for uncacheable response codes (RFC 7231 section 6.1),
    // This implementation doesn't support caching partial content.
    switch (response.code()) {
      case HTTP_OK:
      case HTTP_NOT_AUTHORITATIVE:
      case HTTP_NO_CONTENT:
      case HTTP_MULT_CHOICE:
      case HTTP_MOVED_PERM:
      case HTTP_NOT_FOUND:
      case HTTP_BAD_METHOD:
      case HTTP_GONE:
      case HTTP_REQ_TOO_LONG:
      case HTTP_NOT_IMPLEMENTED:
      case StatusLine.HTTP_PERM_REDIRECT:
        // These codes can be cached unless headers forbid it.
        break;

      case HTTP_MOVED_TEMP:
      case StatusLine.HTTP_TEMP_REDIRECT:
        // These codes can only be cached with the right response headers.
        // http://tools.ietf.org/html/rfc7234#section-3
        // s-maxage is not checked because OkHttp is a private cache that should ignore s-maxage.
        if (response.header("Expires") != null
            || response.cacheControl().maxAgeSeconds() != -1
            || response.cacheControl().isPublic()
            || response.cacheControl().isPrivate()) {
          break;
        }
        // Fall-through.

      default:
        // All other codes cannot be cached.
        return false;
    }

    // A 'no-store' directive on request or response prevents the response from being cached.
    return !response.cacheControl().noStore() && !request.cacheControl().noStore();
}

我们接着回到maybeCache()方法往下看,最后一句responseCache.put(stripBody(userResponse))将会调用下面的方法:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
okhttp3.Cache:
private CacheRequest put(Response response) throws IOException {
    String requestMethod = response.request().method();

    if (HttpMethod.invalidatesCache(response.request().method())) {
      try {
        remove(response.request());
      } catch (IOException ignored) {
        // The cache cannot be written.
      }
      return null;
    }
    if (!requestMethod.equals("GET")) {
      // Don't cache non-GET responses. We're technically allowed to cache
      // HEAD requests and some POST requests, but the complexity of doing
      // so is high and the benefit is low.
      return null;
    }

    if (OkHeaders.hasVaryAll(response)) {
      return null;
    }

    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
      editor = cache.edit(urlToKey(response.request()));
      if (editor == null) {
        return null;
      }
      //缓存写入文件:
      entry.writeTo(editor);
      return new CacheRequestImpl(editor);
    } catch (IOException e) {
      abortQuietly(editor);
      return null;
    }
}

总的来说就是只要是Get请求,同时请求和响应头部CacheControl没有设置no store就会把请求结果缓存了。

OkHttp保存缓存的格式:

url所对应的缓存文件名规则,不难看出是url的md5值。

1
Util.md5Hex(request.url().toString())

保存的代码很明显就在entry.writeTo里面啦。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
okhttp3.Cache.Entry:
public void writeTo(DiskLruCache.Editor editor) throws IOException {
  //.0结尾的文件
  BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));

  sink.writeUtf8(url);
  sink.writeByte('\n');
  sink.writeUtf8(requestMethod);
  sink.writeByte('\n');
  sink.writeDecimalLong(varyHeaders.size());
  sink.writeByte('\n');
  for (int i = 0, size = varyHeaders.size(); i < size; i++) {
	sink.writeUtf8(varyHeaders.name(i));
	sink.writeUtf8(": ");
	sink.writeUtf8(varyHeaders.value(i));
	sink.writeByte('\n');
  }

  sink.writeUtf8(new StatusLine(protocol, code, message).toString());
  sink.writeByte('\n');
  sink.writeDecimalLong(responseHeaders.size());
  sink.writeByte('\n');
  for (int i = 0, size = responseHeaders.size(); i < size; i++) {
	sink.writeUtf8(responseHeaders.name(i));
	sink.writeUtf8(": ");
	sink.writeUtf8(responseHeaders.value(i));
	sink.writeByte('\n');
  }

  if (isHttps()) {
	sink.writeByte('\n');
	sink.writeUtf8(handshake.cipherSuite().javaName());
	sink.writeByte('\n');
	writeCertList(sink, handshake.peerCertificates());
	writeCertList(sink, handshake.localCertificates());
	// The handshake’s TLS version is null on HttpsURLConnection and on older cached responses.
	if (handshake.tlsVersion() != null) {
	  sink.writeUtf8(handshake.tlsVersion().javaName());
	  sink.writeByte('\n');
	}
  }
  sink.close();
}

我们打开相应的缓存目录看一下，果然其中包含了以url的md5值作为文件名的一系列文件，还有一个journal文件，这个文件和DiskLruCache缓存有关，我们这里不展开分析。我们看一下.0和.1文件的内容吧，其实.0里面的东西从上面的代码已经可以知道就是和请求和响应头部的一些信息。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
http://www.infoq.com/cn/articles/etags
GET
2
Accept-Encoding: gzip
User-Agent: okhttp/3.2.0
HTTP/1.1 200 OK
15
Date: Sun, 11 Dec 2016 14:50:36 GMT
Server: Apache
Sniply-Options: BLOCK
Set-Cookie: JSESSIONID=7FFBDE344A66525EE343DCCC6DD23692; Path=/
Last-Modified: Sun, 11 Dec 2016 14:50:36 GMT
Vary: Accept-Encoding,User-Agent
Access-Control-Allow-Credentials: true
Accept-Ranges: none
Access-Control-Allow-Origin: http://www.infoq.com
Content-Encoding: gzip
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html;charset=utf-8
OkHttp-Sent-Millis: 1481467841870
OkHttp-Received-Millis: 1481467844412

md5(url).1文件很容易猜到就是response本身的内容了。读者可以自行验证，另外如果response是gzip的格式，那么这里缓存的直接就是gzip后的内容了。

OkHttp离线缓存实现:

通过以上的分析，我们的思路就很清晰了，其实就是一开始就判断是否连接网络，如果是否就直接返回缓存的内容，根据response的头部是否是Content-Encoding: gzip来确定是否需要解压后返回；如果请求服务端失败，那么也返回缓存的内容。

通过分析OkHttp的InterceptorChain实现：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
okhttp3.RealCall.ApplicationInterceptorChain:

public Response proceed(Request request) throws IOException {
      // If there's another interceptor in the chain, call that.
      if (index < client.interceptors().size()) {
        Interceptor.Chain chain = new ApplicationInterceptorChain(index + 1, request, forWebSocket);
        Interceptor interceptor = client.interceptors().get(index);
        Response interceptedResponse = interceptor.intercept(chain);

        if (interceptedResponse == null) {
          throw new NullPointerException("application interceptor " + interceptor
              + " returned null");
        }

        return interceptedResponse;
      }

      // No more interceptors. Do HTTP.
      return getResponse(request, forWebSocket);
}

不难发现只要通过一个Interceptor返回response即可中断http后续的请求动作，如果服务端错误，response将会返回相应的错误码，总结最后的实现代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
final class CacheControlInterceptor implements Interceptor {

    private final String mCacheDir;

    private final ILog mLog;

    CacheControlInterceptor(ILog log, String cacheDir){
        this.mCacheDir = cacheDir;
        this.mLog = log;
    }

    @Override
    public Response intercept(Chain chain) throws IOException {
        Request request = chain.request();
        boolean networkAvail = NetUtils.isNetworkAvailable();
        if (!networkAvail){
            Response cacheResponse = getCacheResponse(request);
            if(cacheResponse != null){
                return cacheResponse;
            }
        }
        Response networkResponse = chain.proceed(request);
        if(networkAvail && !networkResponse.isSuccessful()){
            Response cacheResponse = getCacheResponse(request);
            if(cacheResponse != null){
                return cacheResponse;
            }
        }
        return networkResponse;
    }

    private Response getCacheResponse(Request request) throws FileNotFoundException{

        if(!"get".equalsIgnoreCase(request.method())|| mCacheDir == null){
            return null;
        }

        String urlMd5 = Util.md5Hex(request.url().url().toString());
        File headerCacheFile = new File(new File(mCacheDir), urlMd5 + ".0");
        Response.Builder cacheResponseBuilder = null;
        if (headerCacheFile.exists()) {
            cacheResponseBuilder = new Response.Builder();
            cacheResponseBuilder.request(request);
            readCacheResponseHeaders(headerCacheFile,cacheResponseBuilder);
        }

        if(cacheResponseBuilder != null){
            Response cacheResponse = cacheResponseBuilder.build();
            File bodyCacheFile = new File(new File(mCacheDir), urlMd5 + ".1");
            Source cacheSource;
            if(!"gzip".equalsIgnoreCase(cacheResponse.header("Content-Encoding"))){
                cacheSource = Okio.source(bodyCacheFile);
            }else{
                cacheSource = new GzipSource(Okio.source(bodyCacheFile));
            }
            RealResponseBody responseBody = new RealResponseBody(cacheResponse.headers(),Okio.buffer(cacheSource));
            return cacheResponse.newBuilder().body(responseBody).build();
        }

        return null;
    }

    private void readCacheResponseHeaders(File headerCacheFile,Response.Builder responseBuilder) {
        BufferedReader reader = null;
        try {
            reader = new BufferedReader(new InputStreamReader(new FileInputStream(headerCacheFile)));
            String lineText;
            int code = 0;
            boolean isHeaderBegin = false;
            while((lineText = reader.readLine()) != null){

                if(isHeaderBegin){
                    int sepIndex = lineText.indexOf(":");
                    String headerName = lineText.substring(0,sepIndex);
                    String headerVal = lineText.substring(sepIndex + 1).trim();
                    responseBuilder.addHeader(headerName,headerVal);
                }

                if(code != 0 && NumberUtils.parseInt(lineText,0) != 0){
                    isHeaderBegin = true;
                    continue;
                }

                if(lineText.startsWith("HTTP/1.1")){
                    code = Integer.valueOf(lineText.substring(lineText.indexOf(" ")).trim());
                    responseBuilder.protocol(Protocol.HTTP_1_1);
                    responseBuilder.code(code);
                }
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            IOUtils.closeSilently(reader);
        }
    }
}

参考资料

  1. http缓存: https://developers.google.cn/web/fundamentals/performance/optimizing-content-efficiency/http-caching?hl=zh-cn
  2. tomcat中对客户端的缓存机制: http://blog.csdn.net/liweisnake/article/details/8524179
  3. Okhttp Interceptors: https://github.com/square/okhttp/wiki/Interceptors
坚持原创分享，您的支持将鼓励我继续创作！