2021年12月1日 星期三

讓我們Py在一起:手勢控制PowerPoint放映切換

讓我們Py在一起:

手勢控制PowerPoint放映切換

Line:ted2016.kpvs
Email:Lct4246@gmail.com
FBhttp://gg.gg/TedLeeFB/
Bloghttp://gg.gg/TedLeeBlog/

Dec. 1, 2021
[1]
 

%3CmxGraphModel%3E%3Croot%3E%3CmxCell%20id%3D%220%22%2F%3E%3CmxCell%20id%3D%221%22%20parent%3D%220%22%2F%3E%3CmxCell%20id%3D%222%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3B%22%20edge%3D%221%22%20source%3D%223%22%20target%3D%228%22%20parent%3D%221%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%223%22%20value%3D%22cvzoneRSP_%E8%A9%B3%E8%A8%BB.py%22%20style%3D%22ellipse%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3Baspect%3Dfixed%3BfillColor%3D%23d5e8d4%3BstrokeColor%3D%2382b366%3BfontSize%3D20%3B%22%20vertex%3D%221%22%20parent%3D%221%22%3E%3CmxGeometry%20x%3D%22-170%22%20y%3D%2295%22%20width%3D%22210%22%20height%3D%22210%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%224%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BexitX%3D1%3BexitY%3D0.5%3BexitDx%3D0%3BexitDy%3D0%3BentryX%3D0%3BentryY%3D0.5%3BentryDx%3D0%3BentryDy%3D0%3B%22%20edge%3D%221%22%20source%3D%225%22%20target%3D%223%22%20parent%3D%221%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%225%22%20value%3D%22Hand%20Tracking.py%22%20style%3D%22ellipse%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3Baspect%3Dfixed%3BfillColor%3D%23d5e8d4%3BstrokeColor%3D%2382b366%3BfontSize%3D20%3B%22%20vertex%3D%221%22%20parent%3D%221%22%3E%3CmxGeometry%20x%3D%22-380%22%20y%3D%22115%22%20width%3D%22170%22%20height%3D%22170%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%226%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BentryX%3D0%3BentryY%3D0.5%3BentryDx%3D0%3BentryDy%3D0%3B%22%20edge%3D%221%22%20source%3D%227%22%20target%3D%228%22%20parent%3D%221%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%227%22%20value%3D%22PPT%20slideshow.py%22%20style%3D%22ellipse%3BwhiteSpace%3Dwrap%3Bhtml%3D1%3Baspect%3Dfixed%3BfillColor%3D%23ffe6cc%3BstrokeColor%3D%23d79b00%3BfontSize%3D20%3B%22%20vertex%3D%221%22%20parent%3D%221%22%3E%3CmxGeometry%20x%3D%22-150%22%20y%3D%22320%22%20width%3D%22190%22%20height%3D%22190%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%228%22%20value%3D%22%26lt%3Bspan%20style%3D%26quot%3Bfont-size%3A%2020px%3B%26quot%3B%26gt%3BPPT%20controller.py%26lt%3B%2Fspan%26gt%3B%22%20style%3D%22whiteSpace%3Dwrap%3Bhtml%3D1%3Baspect%3Dfixed%3BfontSize%3D20%3B%22%
在我們在先前的拙著《Python玩AI,你也可以》(https://reurl.cc/Kr1ADM)中,成功的解析了「剪刀石頭布RSP」實例(RSP.py)。緊接著,我們也測試了PowerPoint檔自動放映的範例(Slideshow.py)。
本文嘗試將這兩隻個別測妥的程式合併成可以「用手勢自動切換PowerPoint」的小專案(PPT controller.py),並以此過程來展示專案開發的技巧:

將大問題拆解為數個小問題後,再將各部份的解答回組成原解。


圖1詳盡地說明本文發展的過程:第一線,和陳會安老師學了CVzone的 ch6-5.py 範例後(https://reurl.cc/gzbqlb),我們先很用功的寫出每一行程式的註解 cvzoneRSP_詳註.py 。接著,再發想出用手勢來控制PowerPoint的播放切換。隨後,著手搜尋並測試了 PPT slideshow.py 這個程式片段。完成之後,我們將這兩隻程式合拼成 PPT controller.py。



圖1:手勢控制PowerPoint切換的演進過程。

cvzoneRSP_詳註.py

詳細說明請讀者參閱《Python玩AI,你也可以》(https://reurl.cc/Kr1ADM),以下僅列出完整程式碼:


from cvzone.HandTrackingModule import HandDetector #L5初始化用
import cv2
cap = cv2.VideoCapture(0) #攝影機的handler
detector = HandDetector(detectionCon=0.5, maxHands=1) #detectionCon:偵測的信心值(confidence)、maxHands:可測到幾隻手
while cap.isOpened(): #當攝影機被開啟時
    success, img = cap.read() #回傳攝影機讀到的影像
    hands, img = detector.findHands(img) #從影像中中偵測手
    if hands: #如果有偵測到手
        hand = hands[0]
        bbox = hand["bbox"] #bbox: bounding box
        fingers = detector.fingersUp(hand) #fingers = [手指1, 手指2, 手指3, 手指4, 手指5]
        totalFingers = fingers.count(1) #fingers裡有個1
        print(totalFingers)
        msg = "None" #顯示剪刀、石頭、布
        
        if totalFingers == 5:
            msg = "Paper"
            
        if totalFingers == 0:
            msg = "Rock"
            
        if totalFingers == 2:
            if fingers[1] == 1 and fingers[2] == 1: #食指 + 中指
                msg = "Scissors"
                
        cv2.putText(img, msg, (bbox[0]+200,bbox[1]-30), cv2.FONT_HERSHEY_PLAIN, 2, (0, 255, 0), 2) #顯示bbox框框 + 訊息
    cv2.imshow("Image", img) #顯示圖片img,視窗標題為Image
    
    if cv2.waitKey(1) & 0xFF == ord("q"): #按q離開迴圈
        break
    
cap.release() #釋放攝影機handler cap
cv2.destroyAllWindows() #關閉視窗

PPT slideshow.py

這篇Satck Overflow中的文章(https://reurl.cc/NZKoYk)展示如下的程式碼,它是援用Python版的Win32擴充(extensions) pywin32https://reurl.cc/pxeRp8) 使用 COM(Component Object Model ,元件物件模型) 來控制PowerPoint(https://reurl.cc/Rb3Z66)。

import win32com.client #L4, 5用
import time #L8, 11, 14, 17用

app = win32com.client.Dispatch("PowerPoint.Application")
presentation = app.Presentations.Open(FileName=u'd:\\要播放的PowerPoint檔.pptx', ReadOnly=1)  #開啟PowerPoint檔

presentation.SlideShowSettings.Run()  #PowerPoint放映
time.sleep(1) #延遲一段時間

presentation.SlideShowWindow.View.Next()  #切換到下一頁
time.sleep(1)

presentation.SlideShowWindow.View.Next()  #切換到下一頁
time.sleep(1)

presentation.SlideShowWindow.View.Previous()  #切回到上一頁
time.sleep(1)

presentation.SlideShowWindow.View.Exit() #關閉PowerPoint檔
app.Quit() #關閉PowerPoint
  
          

讓我們Py在一起:PPT controller.py

cvzoneRSP_詳註.py 和 PPT slideshow.py 整併如下:

import win32com.client
import time

app = win32com.client.Dispatch("PowerPoint.Application")
presentation = app.Presentations.Open(FileName=u'd:\\Bit It.pptx', ReadOnly=1)

presentation.SlideShowSettings.Run()

delay_find = 0
flag = 0

import time

from cvzone.HandTrackingModule import HandDetector
import cv2

cap = cv2.VideoCapture(0) #使用第一台攝影機
detector = HandDetector(detectionCon=0.5, maxHands=1) #手部偵測

#攝影机已開啟
while cap.isOpened():
    success, img = cap.read() #讀入影像
    hands, img = detector.findHands(img) #找手
    
    if flag == 0:       
        #找到手
        if hands:
            hand = hands[0]
            bbox = hand["bbox"]
            
            #有幾根手指
            fingers = detector.fingersUp(hand)
            totalFingers = fingers.count(1)
            msg = "None"
            if totalFingers == 1:
               presentation.SlideShowWindow.View.Next()
            
            cv2.putText(img, msg, (bbox[0]+200,bbox[1]-30), cv2.FONT_HERSHEY_PLAIN, 2, (0, 255, 0), 2)                                       
            flag = 1
            delay_find = 0
    cv2.imshow("Image", img)
    
    delay_find = delay_find + 1    
    if delay_find > 30:
       flag = 0
       
#按q離開
    if cv2.waitKey(1) & 0xFF == ord("q"):
        presentation.SlideShowWindow.View.Exit() #關閉PowerPoint
        app.Quit() #關閉PowerPoint
        break
        
cap.release()
cv2.destroyAllWindows()

延伸應用發想

可以在控制PowerPoint放映時還能像新聞主播播報新聞那樣隨時圈畫重點嗎?也許將本文再結合這篇「AI虛擬畫家(https://tinyurl.com/y2jxsdo4 」就能達成。讀者們可以自行試試看。或者,電腦視覺特區(Computer Vision Zone)https://tinyurl.com/y5msr5pa)上也有許多使用AI的電腦視覺專案,像是「虛擬鍵盤(Virtual Keyboard)」,有興趣的讀者可以參考它們的程式碼再行整併出更有趣的實例。
期待!

後記:如何解題?

一般而言,解決問題的基本方法論(methodology)有兩種:由上而下(Top Down
由下而上(Bottom Up,如圖2和圖3所示。
前者是把大問題先拆解為各個功能獨立的小問題,個別求出其解後再將之組成原問題的解答;後者是從已知解答的各個小問題再發想出相關的應用,而這個新問題的答案就能快速地從已知合併(merge)出最終的結果。

圖2:由上而下求解。


圖3:由下而上求解。

然而,如果讀者已熟悉圖2及圖3的方法了,多練習幾個範例以建立直覺(intuition)。之後,再把這些方法都數忘記。不過,到那時,相信您定可信手拈來,而雖離師輔,道亦不遠矣!

圖4:全方位求解。