Reading it in mBlock 5

Warning

v1 not yet hardware-verified: the value layout is final and passes desktop tests; real-world numbers (fps, line thresholds) may change after hardware testing. The mBlock edition has only been tested on a PC (replies for every value id, E0/E1, time-outs) and has not been tried on an mBot yet.

The mBlock edition firmware of the SharpShot Vision lets mBot, mBot Ranger, Orion and MegaPi read the camera with mBlock 5 blocks: the MBC Pro Sensors extension v2.2.0 has a "SharpShot Vision" category (37 blocks: the 22 of v2.1.0 + 15 for camera mode 5 object detection; the old school / WRO edition category is now called "SharpShot Vision (legacy)"). The value table is exactly the one of the Arduino I2C edition, carried over the RJ25 serial link instead; the blocks first ask the camera which mode it is in, then read the right value.

Wiring

The camera must run the mBlock edition firmware (a different .bin from the SPIKE / EV3 / MATRIX / I2C editions; ask us for a camera flashed with it).

• Serial 38400 bps, 8N1; camera RX = IO3, TX = IO1, crossed to the host (the socket board already does this: S1 = host TX, S2 = host RX).
• The mBot2 (CyberPi) has no RJ25 port and is not supported.

Caution

Wiring a 5 V Arduino directly (no socket board): the UNO / Mega TX is 5 V and the ESP32 pins only take 3.3 V, so put a divider (e.g. 1 kΩ + 2 kΩ) between board TX and camera RX; camera TX → board RX can be wired directly.

Install the extension

1. New extension in development: MBC Pro Sensors extension v2.2.0 is still being developed; before it is officially listed you can download the import file to try it (not yet hardware-verified). Once listed it will be in the mBlock 5 Extension Center (search "MBC") and this page will be updated.
2. Try it with the import file: download MBC_Pro_Sensors_mBlock5_v2.2.0_import.mext → sign in to ext.mblock.cc with your own Makeblock account → My extensions → Import data → upload (about 2 minutes; stay on the page until it reports success) → click "Download" to get a .mext you can drop into mBlock.
3. Open mBlock 5 desktop, add your board under "Devices" (bottom left, e.g. mBot), drop the .mext into the mBlock window; a "SharpShot Vision" category appears (next to the other MBC categories).
4. Switch the mode toggle (bottom right) to Upload: the extension only works in Upload mode (the program runs on the board).

The other categories of v2.2.0 (Pathfinder, Ranger, Controller, IMU, SharpShot Vision (legacy) = school / WRO edition) have the same blocks as v2.0.0 (the legacy category only got a new display name); see Makeblock extension.

Set the camera mode first

The same value id means different things in different camera modes (0x10 is colour 1 x in colour tracking but near offset in line following), so start the program with "set mode", or pick the mode in the camera menu (MODE). Every reporter first reads the mode number (value 0x01) (reusing the last one for 100 ms) and returns −32768 in the wrong mode.

Target colours can only be taught on the camera (stop on target 1-16 and press OK); programs cannot change them. Colour-pick sample points can be moved with the "set sample point" block (stored in the camera).

Blocks

Pick the port the sensor is actually plugged into. Colour c = target c on the camera (from 1, the same number as on the camera screen); k = values per colour (per point) in that sub-option: colour tracking 1 → 2, 2 → 3, 3 → 4, 4 → 6; colour pick 1 → 1, 2 → 2, 3 → 3; object detection 1 → 4, 2 → 10 (object n = the object with the n-th largest area).

• −32768 means: no reply (time-out, checksum error, unplugged) or the current mode / sub-option has no such value (e.g. 1 max colours has no width / height). Booleans are false then.
• Slot order inside one colour: 1 X Y; 2 X Y count; 3 X Y area count; 4 X Y width height count area (the order of value reference: I2C).
• Out-of-range arguments are clamped (colour 0 → 1, colour 99 → 16; object 0 → 1, object 99 → 8).

The 14 options of "line [line value ▾]"

Object detection (camera mode 5, v2.2.0)

The camera treats every blob of target colours 1-8 as an "object" and judges its shape from the outline. At most 8 per frame, largest area first (object 1 = largest); slots without an object read 0. Colours 9-16 still work for colour tracking, just not as objects.

• 1 object list (51): 4 values per object (k = 4): colour, shape, X, Y; object count at 0x30.
• 2 object details (52): 10 values per object (k = 10): colour, shape, X, Y, width, height, area %, angle, vertices, tracking id; object count at 0x60.
• Fields that 1 does not have (width, height, area, angle, vertices, tracking id) read −32768 in 51; switch to 52 to use them.
• Code = colour × 10 + shape (e.g. 31 = colour 3 circle, 13 = colour 1 triangle; 0 = no such object).
• X, Y are the centroid (colour tracking X, Y are the bounding-box centre, which differs); width / height = bounding box (right − left + 1, bottom − top + 1). Angle 0-179: long side of the minimum bounding rectangle, pointing right = 0, turning towards the bottom of the picture is positive. Tracking id 1-255: stays the same for one object across frames.
• The n in "object (n)" is an input dropdown: pick 1-8 or drop in a variable, e.g. the result of "[colour 1] [triangle] object number".
• The colour and shape menus both have "any": "[any colour] [triangle] object seen?" = is there any triangle; "object (n) is [any shape]?" = does object n exist.

How it talks

• Read: the host sends 1 byte [id] → the camera replies [low][high][id ^ low ^ high] (int16 little-endian); unknown ids reply [00][80][id^80] (−32768). The library waits 10 ms.
• Write: E0 [camera mode, sub-option], E1 [point, X, Y]; the camera ACKs [00][00][id] only after all parameters arrived, then runs it; an incomplete command is dropped after 20 ms. EF ACKs and reboots; there is no F8 (re-addressing only matters for I2C).
• The FD pack-all reply is 235 bytes (116 values), more than the Arduino UNO 64-byte serial buffer, so the extension has no "update all values" block: each block asks only for the value it needs (about 1 ms per value).
• Shared values: F0 device id = 0x034C (what "sensor OK?" checks), F1 protocol 2, F2 firmware 100, F3 status flags (bit 0 ready = booted and camera running, bit 1 something found in the current mode).

Smallest program

Switch to colour tracking 3 standard; target 1 in view → green LED (brighter further right), otherwise dim red:

when mBot starts up :: events hat
wait (2) seconds
SharpShot Vision port [1 v] set mode [colour tracking 3 standard v] :: #E8590C
forever
  if <SharpShot Vision port [1 v] colour [1 v] found? :: #E8590C> then
    LED [all v] colour red (0) green (SharpShot Vision port [1 v] colour [1 v] x :: #E8590C) blue (0) :: looks
  else
    LED [all v] colour red (40) green (0) blue (0) :: looks
  end
end

// SharpShot Vision × mBot: smallest reading sketch (does what the blocks above do, Arduino IDE version)
// Needs the Makeblock library + MBC_Sensors v2.2.0 (the four files in src/ of the extension .mext, in Documents/Arduino/libraries/MBC_Sensors/)
#include <MeMCore.h>
#include <SoftwareSerial.h>
#include <MBC_Sensors.h>

MBCPort cam(1);                 // camera on mBot port 1
MeRGBLed rgbled(7, 2);

void setup() {
  cam.begin();
  delay(2000);                  // let the camera boot
  ucSetMode(cam, 13);           // colour tracking 3 standard (E0 [1, 3])
  while (1) {
    if (ucFound(cam, 1)) {      // colour 1 found?
      int x = ucBlob(cam, 1, 0);          // colour 1 x (0~159)
      rgbled.setColor(0, 0, x, 0);        // brighter green the further right
    } else {
      rgbled.setColor(0, 40, 0, 0);       // not found: dim red
    }
    rgbled.show();
  }
}

void loop() {}

Smallest object-detection program: switch to object detection 1 object list; a colour 1 triangle in view → green LED, otherwise red (the full "chase the red triangle" is mBlock beginner example 6):

when mBot starts up :: events hat
wait (2) seconds
SharpShot Vision port [1 v] set mode [object detection 1 object list v] :: #E8590C
forever
  if <SharpShot Vision port [1 v] [colour 1 v] [triangle v] object seen? :: #E8590C> then
    LED [all v] colour red (0) green (255) blue (0) :: looks
  else
    LED [all v] colour red (255) green (0) blue (0) :: looks
  end
end

// SharpShot Vision × mBot: green LED when a colour 1 triangle is seen (object detection 1 object list)
#include <MeMCore.h>
#include <SoftwareSerial.h>
#include <MBC_Sensors.h>

MBCPort cam(1);                 // camera on mBot port 1
MeRGBLed rgbled(7, 2);

void setup() {
  cam.begin();
  delay(2000);                  // let the camera boot
  ucSetMode(cam, 51);           // object detection 1 object list (E0 [5, 1])
  while (1) {
    if (ucHasObj(cam, 1, 3)) {  // a colour 1 triangle (shape 3)?
      rgbled.setColor(0, 0, 255, 0);
    } else {
      rgbled.setColor(0, 255, 0, 0);
    }
    rgbled.show();
  }
}

void loop() {}

More examples (hear the colour, follow a blob, stop when close, follow a black line, chase the red triangle): mBlock beginner examples.

Without mBlock: a plain Arduino board

The MBC_Sensors library inside the extension also works in the Arduino IDE: unzip the .mext, put the four files of src/ (MBC_Sensors.h/.cpp, mbc_proto.h, mbc_ids.h) into Documents/Arduino/libraries/MBC_Sensors/, and give MBCPort any serial port with begin(serial). The functions match the blocks one to one (ucSetMode, ucBlob, ucFound, ucSpot, ucOrderHue, ucLine, ucObj, ucObjFind, ucHasObj, ucValue, ucOk…). For Arduino without mBlock, the I2C edition is faster (16 values per burst read).

// SharpShot Vision (mBlock firmware) on a plain Arduino serial port (no mBot): prints line near offset and angle
// Needs MBC_Sensors v2.2.0 (as above; no Makeblock library). UNO uses software serial D10/D11; Mega uses Serial1 (D19 RX, D18 TX)
#include <SoftwareSerial.h>
#include <MBC_Sensors.h>

MBCPort cam;                    // no port number: begin(serial) below
#if defined(__AVR_ATmega2560__)
#define CAM_SERIAL Serial1
#else
SoftwareSerial camSerial(10, 11);   // RX = D10 (to camera TX = IO1), TX = D11 (to camera RX = IO3, through a divider)
#define CAM_SERIAL camSerial
#endif

void setup() {
  Serial.begin(115200);
  CAM_SERIAL.begin(38400);
  cam.begin(CAM_SERIAL);
  delay(2000);
  Serial.println(ucOk(cam) ? "SharpShot Vision OK" : "No reply: check wiring and that the firmware is the mBlock edition");
  ucSetMode(cam, 41);           // line follow 1 black line
}

void loop() {
  Serial.print("near offset ");
  Serial.print(ucLine(cam, 0));
  Serial.print("  angle ");
  Serial.print(ucLine(cam, 3));
  Serial.print("  junction ");
  Serial.println(ucLine(cam, 6));
  delay(100);
}

FAQ

• Values stay at −32768? → Check "sensor OK?" first: false = wrong port, socket board loose, or the camera is not running the mBlock firmware; true = wrong mode, read "mode number" to see which mode the camera is in (colour tracking 3 = 13, black line = 41, object list = 51).
• No "SharpShot Vision" category, or no object blocks? → You loaded an older extension (in v2.1.0 the category was called "Vision Universal" and had no object detection; before v2.1.0 it did not exist); install v2.2.0 as above.
• Object width, height or angle read −32768? → The camera is in 1 object list (51); those fields only exist in 2 object details (52).
• I changed the mode in the camera menu; how soon do the blocks follow? → Within 0.1 s (the mode-number cache time).
• Always "not found"? → The target colour is not taught on the camera, or the lighting changed a lot; check that the camera screen boxes the blob.
• Can I use Live mode (no upload)? → No, the extension only supports Upload mode.

More

• SharpShot Vision product page
• Value reference: I2C (the same table as mBlock)
• mBlock beginner examples
• MBC_Pro_Sensors_mBlock5_v2.2.0_import.mext