mimi ai pet

Question

Desktop pets blessed by AI.
Unlike OpenAI, it can do anything you want it to do, like download music, draw a picture

the github url : https://github.com/aiaimimi0920/mimi/tree/main

Sam

Answer 1

Can you provide an English translation for the README file?

Viceroy

Answer 2

I don't get what this is supposed to be. I GUI frontend to ChatGPT? Also that repository consists only of two readmes.

Rita

Answer 3

That's why I asked for a translation of the README. I thought it might provide a clue.

Viceroy

Answer 4

There is an "English" button which links to itself, currently. Looks like the repository needs a little more time to get ready.

Gradualism

Answer 5

https://github.com/aiaimimi0920/mimi/issues/1 🙂

Viceroy

Answer 6

I'm editing the contents of the repository, it should be done soon

Sam

Answer 7

Thank you for your help. I have completed the submission of the first version.

I have encountered a small problem. Is there a solution？

It sees that Windows Defender will report a virus It should be caused by me using RCEDIT to modify the icon Is there any way to solve it?
i think this problem like this https://github.com/electron/rcedit/issues/101

Sam

Answer 8

Thanks for providing the English README.

I see that the project is Windows-only. I use Linux, and will not be able to use it.

Viceroy

Answer 9

Caseworm I would suggest that you should upload the source code to the repository as soon as you can as well as provide links to its dependencies, the "free_ai_adapter" plugin and the "free_vector_adapter" plugin. There is no way that I'm going to trust a binary executable that downloads several plugins onto my OS when its source code isn't even available. I think presently you may even be in violation of GitHub's TOS.

Gradualism

Answer 10

Yes, I am preparing to upload it because I need to delete some code that may leak my privacy.

Sam

Answer 11

Rags Stygian Leeanneleeboard
I have submitted all the code, including the main and plugin branches.

You should be able to use the exe in the release package to complete the chat now. Please note that during the first chat, you will download 150MB+ of plugin free_ai_adapter from the cloud storage may cause a slight delay in response.

I have changed all Chinese comments in the code to English, but there may be translation issues with the comments

This is my first time using Godot to complete my practical ideas. I have to say that this journey is really complex and requires perseverance.
During the process, I learned about Godot, Chroma, Playwright, GPT4free, Rainybot, AWS deployment, server code, FastAPI, and more.
The sense of achievement in completing this MIMI pet in the end still brings me joy

In summary, I wish you a happy time playing

Sam

Answer 12

I lol'd.

Shepperd

Answer 13

A video to show it all in action from start to finish would be nice because... to be completely honest: Personally I would not download an exe version of this and I trust this not one bit. Especially considering it is downloading additional code from the internet.

I might be completely wrong here and maybe that's just my general mistrust of anything AI-related speaking but something about this looks fishy to me. But maybe I am just the only one.

Rita

Answer 14

Thank you for your suggestion. I will try to create a video to showcase the content

However, downloading plugin packages from the internet is indeed a necessary process for this project, and I cannot change this because it is precisely because of this that I am able to make MiMi pet support unlimited functionality

Because it can match the best method and download this plugin, which will be loaded into the Godot engine and executed,

But it has to be said that the security issues you pointed out do exist.
Currently, all plugins are developed by me, so there are no security issues. You can discover all the code of the plugin through the plugin branch in the repository.

However, the plugins submitted by others in the future may not necessarily disclose the source code, so there may be issues. This is what I referred to as a security issue in the roadmap. There may be a certain plugin review mechanism before the plugins can be officially released and used by others.

Of course, I understand your doubts about the security of my exe and plugins, as I am not a big company like Google or Apple, and cannot endorse the security of the developed applications.

Sam

Answer 15

Of course, I understand your doubts about the security of my exe and plugins, as I am not a big company like Google or Apple, and cannot endorse the security of the developed applications.

Can the .exe be built from the source code in the repository?

There's only a security issue if users need to download executable code without the ability to see all the source code.

The Godot Engine isn't distributed by a big company like Google or Apple, but it's trusted because anyone can view and download all the source code and build the executable himself.

Viceroy

Answer 16

Yes, you can build exe and plugin packages from source.

But because if you turn on the smart plug-in switch at runtime. Then the most matching plug-in package that has been allocated will be downloaded from the Internet, and then the logic in the plug-in package will be executed.
I believe he is more worried about whether there are security vulnerabilities in the smartly downloaded plug-in packages, or whether it is possible to download malware plug-in packages.

But I can't avoid automatic downloading. If a pop-up window pops up before each download step to let the user choose whether to allow the download, it will interrupt the user experience.

Let me explain the download logic:

1. When I ask a question to AI: "I want to download music xxx"
2. Will be sent to the 3 best plugins on the platform that match this problem
3. Send the plug-in method and question to ai:
   1. If ai determines that there is a plug-in that can execute this logic, it will return the method name and parameters that the plug-in needs to call. Subsequently, the godot engine will download the plug-in and load it into the runtime. Then call it based on the plug-in method name and plug-in parameters
   2. If it is judged that there is no executable plug-in, the problem will be treated as a dialogue.

It is true that the plug-in you download automatically in 3.1 may have security issues. Even if you can get all the source code of the plug-in, it is impossible for you to understand the code of the plug-in every time before downloading to allow the plug-in package to be downloaded.
So I may use a certain audit mechanism to ensure the security of the plug-in package.

Sam

Answer 17

Yes, you can build exe and plugin packages from source.

Where are the instructions for doing that?

Viceroy

Answer 18

you can use godot editor(the editor is in tools branch)-> export->export the exe

you can use "/packing_bot generate_plugin_file your_plugin_name" export a plugin package. the package will in "C:\Users\user_name\AppData\Roaming\Godot\app_userdata\MiMi\plugin_file\packing_bot"

Sam

Answer 19

I will consider adding a pattern in the future:
In this mode, packages will not be automatically pulled from the network,
Allow you to manually manage local plugins,
We will only discover from your local plugin whether there is a method to handle your current question.
I believe this will be helpful for security sensitive users to a certain extent

Sam

Answer 20

security sensitive users

You mean people with common sense.

No offense, but you registered on GitHub and this site five days ago, and you're surprised that people are cautious about downloading your executable code?

Viceroy

Answer 21

The reason for the new registration is because I am in China and you know that if you are in China, you need to be careful to hide your personal information

Actually, I have another account on this forum that has been registered for a long time and often answers questions

Sam

Answer 22

Updated a version, v1_0_1

The main update of this version is the import of the ffmpeg module through gdextension,
allowing you to freely play any format of audio and video, including HTTP streaming
(The main reason for implementing this plugin is that my music plugin often downloads music in incompatible formats such as M4A, so I implemented this plugin)

The audio can also be played normally, but GIF cannot record sound and the demonstration is meaningless.

Sam

Answer 23

The content planned for the next version update is:

1. Download plugin update: The current HTTP form download is too slow

2. Ticket grabbing plugin update: This is a unique feature of China, where you need to grab tickets during the Chinese New Year

Sam

Answer 24

Where does all the music and video come from, that the AI can search for and download?

Rita

Answer 25

1. From many different sources, such as QQ Music, NetEase Music, music resource sites, or from search engines
2. Then from the searched links, match the song that best matches your needs based on the artist name and song title.
3. download it and play

Sam

Answer 26

Is that legal?

Rita

Answer 27

I am not very familiar with the laws of other countries, but in China, it is legal for you to download music from QQ Music and NetEase Music. AI plugins only help you search for the most suitable music for you and download it. In theory, you can manually download the same music from these sources
As for downloading music from resource websites, it will not constitute infringement in China

As for foreign laws, I am not very familiar with them.

In short, music_adapter plugin is Borrowed from this python module https://github.com/charlespikachu/musicdl.
You can check if musicdl has any infringement issues

Sam

Answer 28

After continuous attempts, I finally successfully loaded aria2.DLL into the GD plugin and completed a download.

It was really complicated, but I am happy to have completed the first step. The next step will be to slightly encapsulate the interface of aria2

Sam

Answer 29

I integrated Aria2 into Godot, and you can get higher download speeds like this

The main purpose of implementing this plugin is to accelerate the download speed of users in MIMI

Sam

Answer 30

update v1_0_2:

1. add aria2 support : You can get faster download speed
2. add downloader_list plugin : You can manage the current download properties and use HTTP and aria for downloading
3. add file_list ui : You can mount cloud storage content such as AWS, Cloudflare, and IPFS, and download these contents

Sam

Answer 31

The content planned for the next version update is:

1. Voice system, I hope to have a conversation with my robot through voice

Sam

Answer 32

I found this plugin to implement the voice system, although it is incomplete: https://github.com/V-Sekai/godot-whisper

In actual testing, it was found that running under Windows would result in an unresponsive state, and the recognition effect was not ideal.

I plan to study how to improve his code, I hope everything goes smoothly

Sam

Answer 33

I didn't expect that it has been 2 weeks since the last update of the post, and TTS and Whisper have been tormenting me endlessly. In the end, I have preliminarily implemented the dialogue plugin,

Godot Whisper: I have made some contributions on the basis of the original code ：https://github.com/V-Sekai/godot-whisper
https://github.com/V-Sekai/godot-whisper/assets/153103332/a1fed48f-59a7-4ae9-9da6-6c0734a14b35

Godot_tts: I used SummerTTS as the core code to implement a Godot plugin: https://github.com/aiaimimi0920/godot-tts

https://github.com/aiaimimi0920/godot-tts/assets/153103332/0813ddc9-2c71-453e-8ddb-50ce7f846ba9

I should make further changes to the implementation of these two plugins in the future and use them in my mimi pet

Sam