Using Surface in Android native code

Well, here I set out deeper to use Surface in the native code. In my search for more information on Surfaceflinger, again I dived deeper into MediaPlayer code only to be drowned once gain, nevertheless, these posts in the Android groups helped me in undertading and narrowing down to ISurface interface.  Thanks to Dave Sparks

http://groups.google.co.in/group/android-framework/browse_thread/thread/d3f242356375effa/44d88801c76fcfd3?hl=en&q=codec&pli=1

All we need is, to make call to three methods, defined in the ISurface interface.

1. registerBuffers

2. postBuffer

3. unregisterBuffer

It sounded easy in the beginning but it to was a quite a task to figure out the usage of many methods, as there were security restrictions to call registerBuffers 

Here, the idea is, Java application will pass the Surface object, created with PUSH_BUFFER  and in the native code, from the Surface object, we get the ISurface handle. Using this handle, we call register buffers and post RGB 565 format buffer, using double buffering.

Again we hack SimpleJNI sample, first the Native code, the required header files

#include "SkBlurMaskFilter.h"

#include "ui/Surface.h"

#include "ui/ISurface.h"

#include "utils/MemoryHeapBase.h"

#include "utils/MemoryBase.h"


Now add required libraries to Android.mk

libsgl  \

libui \

libsurfaceflinger

In JNI, we need to use Fields to retrieve the class objects and arguments passed in the JNI calls. Casting directly does not work, so here we declare a fieldID to get the surface instance  

static jfieldID gSurface_SurfaceFieldID;

Global variables for surface and frame buffer rendering

int frame_offset[2];

int buf_index;

short *frame_buf;

int width, height;

sp isurface = 0;

sp frame_heap;

Add new JNI method, this method is called from the Java application in the setSurfacChanged. 

static jint

setSurfaceChanged(JNIEnv *env, jobject thiz, jobject surf, jint w, jint h) {

Surface* const p = (Surface*) env->GetIntField(surf, gSurface_SurfaceFieldID);

const sp& native_Surface = sp(p);

sp native_iSurface = MediaPlayer::getISurface( native_Surface);

registersurface(native_iSurface,w,h);

}

Now we have the ISurface handle, we can go ahead and call those three methods. Here, I want highlight the  ISurface::getISurface  which is actually a private method but accessible through MediaPlayer as it is declared friend in the ISurface class. So, we declare a dummy MediaPlayer class and call ISurface::getISurface

Things did not work as expected until I stumbled upon http://groups.google.com/group/prajnashi, who is maintaining gst-plugin-android. There is an implementation of wrapper in folder sink that I used here to understand the Android Surface architecture, esp borrowed code in postBufers

//------------------------------------------------------------------------------------------------------------------

/* store the ISurface handle in a global variable, then we create MemoryHeapBase heap, use it create Frame buffer, then call registerBuffers

*/

void registerBuffer(sp& isurf, int w, int h){

  buf_index = 0;

width = w;

height = h;

   for ( int i = 0; i&MAX_FRAME_BUFFERS; i++){

frame_offset[i] = 0;

    }

    

isurface = isurf;

    /* use double buffer in post */

    int frameSize = width * height * 2;

    /* create frame buffer heap base */

    frame_heap = new MemoryHeapBase(frameSize * MAX_FRAME_BUFFERS);

    

if (frame_heap->heapID() < 0) {

      LOGI("Error creating memory base  heap!");

return;

    }

    /* create frame buffer heap and register with surfaceflinger */

    ISurface::BufferHeap buffers( (width + 1)& -2,(height + 1)  & -2 ,width,height, 

PIXEL_FORMAT_RGB_565, frame_heap);

    if (isurface->registerBuffers(buffers) < 0 ){

        LOGI("Cannot register frame buffer!");

        frame_heap.clear();

return;

     }

LOGI("Registered surface! %d, %d",width, height);

   for ( int i = 0; ii&MAX_FRAME_BUFFERS; i++){

        frame_offset[i] = i*frameSize;

    }

    buf_index = 0;

frame_buf = new short[width*height];

}

//--------------------------------------------------------------------------------------------------

/*

This is the JNI method called from SurfaceView::OnDraw. Here I use frame_buf to set some random values, then call postBuffer

*/

static jint

draw(JNIEnv *env, jobject thiz, jint n) {

if (++buf_index == MAX_FRAME_BUFFERS) 

buf_index = 0;

   

for (int i=0; i< width*height; i++){

frame_buf[i] = (short)(i+n*16) ;

}

memcpy (static_cast<unsigned char *>(frame_heap->base()) + frame_offset[buf_index],  frame_buf, 

width*height*sizeof(short));

    

isurface->postBuffer(frame_offset[buf_index]);

return 1;

}

//--------------------------------------------------------------------------------------------------

/* JNI method for final cleanup called from surfaceDestroyed */

static jint 

destroySurface(JNIEnv *env, jobject thiz, jint i){

isurface->unregisterBuffers();

LOGI("Unregistered surface!");

if(frame_buf)

free(frame_buf);

frame_buf = NULL;

isurface.clear();

LOGI("destroySurface done!");

return 1;

}

//--------------------------------------------------------------------------------------------------

Here is Java code changes in SimpleJNI.java  

public void surfaceChanged(SurfaceHolder holder, int format, int width,

         int height) {

    thread.setSurfaceSize(width, height);

Native.setSurfaceChanged(holder.getSurface(), width,height);

}

//--------------------------------------------------------------------------------------------------

There is one more hurdle to overcome, that is security, I did not want to bypass security, I tried to sign with platform key but still I got “access denied”, then I added   “dangerous” to frameworks/base/core/res/AndroidManifest.xml. still got  Permission failure: android.permission.ACCESS_SURFACE_FLINGER from uid=10012 pid=774 then I had no choice but to comment out the checkCallingPermission. Finally everything worked to my glee!