OpenCV Fundamentals

Single Channel Display in OpenCV

When extracting and displaying a single B, G, or R channel from a full-color image, the result appears as a grayscale image. This is not actually a grayscale representation—it uses black, white, and shades of gray for display purposes. The intensity corresponds to the channel's value: brighter areas indicate more of that channel's color. For instance, when viewing the red channnel, white represents maximum red intensity while black represents the absence of red. This visualization approach exists because grayscale images are easier for human eyes to interpret.

BGR vs RGB Color Space in OpenCV

OpenCV stores color images in BGR format, where channels are ordered blue, green, then red. This differs from RGB ordering used elsewhere. When displaying a normal-looking image in OpenCV, the input image must be in BGR format to match the expected channel order. If you pass an RGB image to OpenCV functions without conversion, the colors will appear swapped because the channel assignments don't align.

Pixel Coordinate Notation with cv::at

OpenCV uses row-major storage for matrices. When accessing pixels with cv::at, specify the row index first (y-coordinate), then the column index (x-coordinate). The correct syntax is Mat::at(y, x), not Mat::at(x, y). This convention reflects how OpenCV stores image data internally, with consecutive memory locations representing subsequent rows.

Understanding Contour Hierarchy with findContours

The hierarchy returned by cv2.findContours describes parent-child relationships between contours. A contour is a closed curve, so a filled shape with an internal hole has two separate contours: the outer boundary (external contour) and the inner boundary (internal contour or hole). A simple filled shape without holes has only one contour. When analyzing contours, count how many closed boundaries are needed to encloce each white region—the number of boundaries equals the number of contours for that region.

ROS Build System Concepts

Header Files vs Library Files

Header files contain function declarations—names, parameter lists, and return types—enabling compile-time type checking and syntax validation. They do not contain function implementations or library locations.

Library files hold compiled function definitions and implementations. The linker matches function references in object files to their definitions in libraries during the build process.

During compilation, the compiler needs declarations to verify code correctness but doesn't require implementation details or library locations. The linker handles connecting compiled code to library implementations. Header files provide compile-time interface information; they don't specify where libraries reside on disk.

CMakeLists: find_package vs catkin_package

The find_package() function locates dependencies needed by the current package. It enables the current package to find headers and libraries from other packages.

The catkin_package() macro, unique to catkin, generates the information other packages need when depending on this one. It defines variables like ${catkin_INCLUDE_DIRS} and ${catkin_EXPORTED_TARGETS} that become available after the macro executes. Therefore, catkin_package() must remain uncommented in CMakeLists.txt even if all its parameters are commented.

Required CMakeLists structure:

cmake_minimum_required(VERSION 3.0.2)
project(your_package_name)
find_package(catkin REQUIRED)
catkin_package()

When adding catkin_DEPENDS to catkin_package(), update package.xml by adding corresponding entries to the <build_export_depend> section.

catkin_package() parameters:

• INCLUDE_DIRS: Paths to headers for other packages
• LIBRARIES: Libraries for other packages
• CATKIN_DEPENDS: Other catkin packages this package depends on
• DEPENDS: Non-catkin packages this package depends on

add_dependencies vs target_link_libraries

target_link_libraries() links compiled libraries and executables. add_dependencies() ensures that custom Messages, Services, and Actions are built before the linking phase occurs. Use add_dependencies() when depending on self-defined message, service, or action files.

Understanding EXPORTED_TARGETS Variables

The ${PROJECT_NAME}_EXPORTED_TARGETS} variable is local to the current package, while ${catkin_EXPORTED_TARGETS} is global. Both contain the same exported target lists, but the local variable is scoped to the specific package's custom messages, services, and actions. The global variable encompasses all packages' exported targets.

For most cases, ${catkin_EXPORTED_TARGETS} alone suffices, but including both is harmless and provides additional safety.

Package.xml Dependency Explanations

The build_export_depend tag specifies packages whose headers are transitively included when depending on this package. For example, if package B depends on A and exports A in its build configuration, then package C depending on B can use A's headers. This transitive header access prevents missing dependency errors.

ROS Namespace Behavior

Node Naming in Launch Files

<launch>
    <node pkg="ssr_pkg" type="yao_node" name="yao" ns="new_name"/>
    <node pkg="ssr_pkg" type="li_node" name="li_node"/>
    <node pkg="atr_pkg" type="keji_node" name="keji_node" output="screen"/>
</launch>

The type attribute specifies the executable name, while name specifies the actual node name used at runtime. The name parameter employs the remapping mechanism—if the code sets the node name to yao_node but launch sets it to yao, the node runs as yao. The name parameter cannot include namespace prefixes.

Terminal remapping syntax also cannot include namespaces: rosrun turtle_tf2 turtle_tf2_broadcaster __name:=turtle1_tf2/turtle1 is invalid. To specify both namespace and name, use: rosrun turtlesim turtlesim_node __ns:=/xxx __name:=tn.

Namespace Effects on Topics

The ns attribute places a node within a namespace. The node yao with ns="new_name" becomes new_name/yao in the topic tree.

Global namespace defaults to "/". Topics with leading slashes ignore namespace remapping:

ros::Publisher pub1 = n.advertise<std_msgs::String>("/chatter", 10);
ros::Publisher pub2 = n.advertise<std_msgs::String>("chatter", 10);

Topics with "/" prefix remain in the global namespace, while topics without prefix inherit the publishing node's namespace. Two nodes in different namespaces can communicate if their topic names are identical and use the global prefix. Otherwise, they create separate topics prefixed with their respective namespaces.

Mathematical Foundations

2D Rotation Matrix Intuition

Any vector in 2D space can be represented using standard orthonormal basis vectors. Think of this as using coordinate axes—except instead of x and y coordinates, you express the vector using 2D basis vectors.

The key insight involves fixed and rotating reference frames. A vector rotates while the rotating frame remains stationary relative to the vector but rotates relative to the fixed frame. By expressing the rotating frame's basis vectors in the fixed frame, you derive the rotated vector's representation.

TF Coordinate Transformation System

TurtleBot Coordinate System

The turtle's head direction defines the x-axis, and the direction perpendicular to the turtle's plane defines the z-axis, following the right-hand rule.

TF Tool Concepts

Coordinate Transformation vs Frame Transformation

Coordinate transformation converts a single point's representation between different frames. Frame transformation describes the relative pose relationship between two coordinate frames.

Source frame and target frame apply to coordinate transformations. The source frame is where the original point exists; the target frame is where you want the point's representation. The transformation T_source^target gives the target frame's representation of a point originally in the source frame.

Parent frame and child frame apply to frame transformations. The parent frame is the original frame; the child frame is the transformed frame. This describes the child frame's origin position in the parent frame's coordinates. This terminology appears when constructing TF broadcasters to establish parent-child relationships.

tf_echo usage: The command rosrun tf tf_echo <target_frame> <source_frame> displays the transformation where the source frame's point appears in the target frame's coordinates.

lookupTransform signature:

void tf::TransformListener::lookupTransform(
    const std::string& target_frame,
    const std::string& source_frame,
    const ros::Time& time,
    StampedTransform& transform
)

The parameter order here correctly reflects the transformation definition.

CMakeLists Install Section

Install Targets and catkin_make install

By default, catkin builds targets into the devel directory. The Install section in CMakeLists packages the project for distribution, optionally creating an installable package without source code.

# Mark executable scripts for installation
install(PROGRAMS
    scripts/talker.py
    scripts/listener.py
    DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)

# Mark executables and libraries for installation
install(TARGETS talker_node listener_node
    ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
    LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
    RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)

# Mark header files for installation
install(DIRECTORY include/${PROJECT_NAME}/
    DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
    FILES_MATCHING PATTERN "*.h"
    PATTERN ".svn" EXCLUDE
)

# Mark resource directories for installation
install(DIRECTORY model urdf mesh rviz
    DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

# Mark other files for installation
install(FILES
    launch/bringup.launch
    DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/launch
)

DESTINATION variables contain relative paths. The full path is constructed as ${CMAKE_INSTALL_PREFIX} + DESTINATION path, where ${CMAKE_INSTALL_PREFIX} defaults to the workspace install folder.

The catkin_make install command executes the Install section. After installation, source the setup.bash from the install directory to test. Remove any conflicting workspace sourcing from .bashrc to verify the install-based execution works independently.

Custom install location:

catkin_make -DCMAKE_INSTALL_PREFIX=/opt/ros/noetic install

This modifies the install prefix from the default workspace install folder to the specified path.

Quaternion Representations

tf2::Quaternion vs geometry_msgs::Quaternion

tf2::Quaternion from the tf2 library represents rotations and orientations. It provides member functions for rotation operations, transformations, and mathematical computations. Include <tf2/LinearMath/Quaternion.h> to use it.

geometry_msgs::Quaternion is a message type storing rotation as four floating-point fields (x, y, z, w). It's a simple data structure without operational methods. Include <geometry_msgs/TransformStamped.h> to use it.

Usage distinction: Use tf2::Quaternion when performing calculations like converting Euler angles to quaternions via setRPY(). Use geometry_msgs::Quaternion for storing and transmitting rotation data.

tf2Scalar Type

tf2Scalar is a typedef for double in the tf2 library. When accessing quaternion components via methods like q.x(), q.y(), q.z(), and q.getW(), these methods return tf2Scalar (double) values.

Direct assignment works between types because geometry_msgs::Quaternion also uses float64 (double) for its components:

transformStamped.transform.rotation.x = q.x();
transformStamped.transform.rotation.y = q.y();
transformStamped.transform.rotation.z = q.getZ();
transformStamped.transform.rotation.w = q.getW();

Note that q.x() and q.getX() are member functions requiring parentheses—they return values, not the values themselves.

Conversion: Include <tf2_geometry_msgs/tf2_geometry_msgs.h> for bidirectional conversion functions between tf2::Quaternion and geometry_msgs::Quaternion.