Flutter: Color each segment of an image independently

I’m working on creating a canvas. The user will be able to select any image from collection and the application will let him/her to color it.

Let’s take an example to make things clear.

A bird: A bird has wings, beak, legs etc. Each of them are individual segments. I need to let the user color each segment independently. So basically if the user swipes say the color red only the background will get colored. So when the user wants to color the beak he’ll have to start from inside beak not anywhere else.

My Solution: Implement a stack. Image will be at the top and the bottom will have a white container. I’ll only let the user color the white container. So GestureDetector will be the parent of the white container. I thought this will let the user color the segments independently. But that didn’t happen and my gesture detector is coloring everywhere when it should only color the container containing the image

My Code:

    return MaterialApp(
      debugShowCheckedModeBanner: false,
      home: Scaffold(
        body: Align(
          alignment: Alignment.centerLeft,
          child: LayoutBuilder(
            builder: (BuildContext context, BoxConstraints constraints) {
              return Row(
                crossAxisAlignment: CrossAxisAlignment.center,
                mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                children: [
                  Flexible(
                    flex: 4,
                    fit: FlexFit.tight,
                    child: Container(
                      height: MediaQuery.of(context).size.height * 0.75,
                      width: MediaQuery.of(context).size.width / 2,
                      child: Container(
                        child: Stack(
                          children: <Widget>[
                            GestureDetector(
                              onPanUpdate: (details) {
                                setState(() {
                                  RenderBox renderBox =
                                      context.findRenderObject();
                                  points.add(TouchPoints(
                                      points: renderBox.globalToLocal(
                                          details.globalPosition),
                                      paint: Paint()
                                        ..strokeCap = strokeType
                                        ..isAntiAlias = true
                                        ..color =
                                            selectedColor.withOpacity(opacity)
                                        ..strokeWidth = strokeWidth));
                                });
                              },
                              onPanStart: (details) {
                                setState(() {
                                  RenderBox renderBox =
                                      context.findRenderObject();
                                  points.add(TouchPoints(
                                      points: renderBox.globalToLocal(
                                          details.globalPosition),
                                      paint: Paint()
                                        ..strokeCap = strokeType
                                        ..isAntiAlias = true
                                        ..color =
                                            selectedColor.withOpacity(opacity)
                                        ..strokeWidth = strokeWidth));
                                });
                              },
                              onPanEnd: (details) {
                                setState(() {
                                  points.add(null);
                                });
                              },
                              child: Container(
                                color: Colors.white,
                              ),
                            ),
                            // Container(
                            //   color: Colors.white,
                            // ),
                            CustomPaint(
                              size: Size.infinite,
                              painter: MyPainter(pointsList: points),
                            ),
                            IgnorePointer(
                              ignoring: true,
                              child: displayImage(context, widget.imagePath),
                            ),
                          ],
                        ),
                      ),
                    ),
                  ),
                  Flexible(
                    flex: 2,
                    fit: FlexFit.tight,
                    child: Container(
                      decoration: BoxDecoration(
                        color: Colors.brown,
                        borderRadius: BorderRadius.all(
                          Radius.circular(20),
                        ),
                      ),
                      height: MediaQuery.of(context).size.height * 0.75,
                      width: MediaQuery.of(context).size.width / 4,
                      child: displayMenu(),
                    ),
                  ),
                ],
              );
            },
          ),
        ),
      ),
    );

I got my motivation of this app from this link.

This is my result until now: