Fyde OS Logo
Enterprise

FydeOS Al User Guide

Last Update: 2024-05-30

What is FydeOS AI?

FydeOS AI is a system-level artificial intelligence assistant specifically tailored for FydeOS. It supports integration with mainstream Artificial Intelligence Generated Content (AIGC) services and can also run local models.

Getting Started with FydeOS AI

System Requirements

Currently, FydeOS AI is in its experimental phase and is only available on openFyde. The supported devices include:

  • openFyde - amd64
  • rpi5-openfyde
  • fydetab_duo-openfyde
  • rock5b-openfyde
  • orangepi5-openfyde
  • edge2-openfyde

Setting Up FydeOS AI

Third-Party API Integration
  1. Click the launcher at the bottom-left corner, locate FydeOS AI within the applications, and launch it.
  2. On the FydeOS AI settings page, under Basic Settings, select your preferred service provider and input the necessary details. Once configured, you can begin using the cloud-based large language model for conversations.
Using FydeOS AI with a Local Model
  1. Visit Hugging Face, choose the desired model, and download the file named XXX.rkllm.
  2. Launch FydeOS AI, navigate to Settings -> Basic Settings, and select the local model option.
  3. Enter the model path. For example, if the model file is in the Downloads folder, the path would be Downloads/xxx.rkllm. If the file is saved elsewhere, use the relative path under "My Files".
  4. Click to initiate the local model and, after a brief wait, it will be ready for conversations.

How to Use FydeOS AI

FydeOS AI is designed to be user-friendly:

  • You can ask FydeOS AI for assistance through the chat box and review previous conversations via the history panel on the left.
  • You can utilise voice input for quick help.
  • When text is selected, press the shortcut Ctrl + C + C (hold Ctrl, then press C twice within two seconds) to copy the content and query FydeOS AI via a pop-up window.

Developer Options

Configuration Files

File Path Description
config/config.yaml Specifies the default parameters for loading the model
config/fix_freq_rk3588.sh Enhances CPU and NPU frequencies when running the model

Parameter Descriptions

Parameter Name Type Description Default
modelPath const char* Path to the model file -
target_platform const char* The hardware platform for running the model, options include "rk3576" or "rk3588" -
num_npu_core int32_t Number of NPU cores to use during inference. For "rk3576", the range is [1, 2]; for "rk3588", the range is [1, 3] -
max_context_len int32_t Sets the context size for the prompt -
max_new_tokens int32_t Sets the upper limit on the number of tokens generated during model inference -
top_k int32_t Top-k sampling selects the next token from the top k most likely tokens predicted by the model. Higher values (e.g., 100) consider more token options, leading to more diverse text; lower values (e.g., 10) focus on the most likely tokens, producing more conservative text. 40
top_p float Top-p sampling, also known as nucleus sampling, selects the next token from a set of tokens with cumulative probability at least p. This balances diversity and quality by considering token probability and the number of tokens sampled. Higher values (e.g., 0.95) result in more diverse text; lower values (e.g., 0.5) produce more focused and conservative text. 0.9
temperature float Controls the randomness of text generation by adjusting the probability distribution of model output tokens. Higher temperatures (e.g., 1.5) make the output more random and creative; lower temperatures (e.g., 0.5) make the output more focused and conservative. 0.8
repeat_penalty float Controls the repetition of token sequences in the generated text, helping to prevent repetitive or monotonous output. Higher values (e.g., 1.5) penalise repetition more strongly, while lower values (e.g., 0.9) are more lenient. 1.1
frequency_penalty float Penalises the use of frequently occurring words/phrases and increases the probability of less common ones. This can make the generated text more diverse but may also lead to incoherent or unexpected results. Range is [-2.0, 2.0]. 0
mirostat int32_t An algorithm that maintains the quality of the generated text within the desired range. It balances coherence and diversity, avoiding low-quality output due to excessive repetition (boredom trap) or incoherence (confusion trap). Values: {0, 1, 2}, with 0 disabling the algorithm, 1 enabling mirostat, and 2 enabling mirostat 2.0. -
mirostat_tau float Sets the target entropy for mirostat, representing the desired perplexity value of the generated text. Lower values result in more focused and coherent text, while higher values result in more diverse but potentially less coherent text. 5.0
mirostat_eta float Sets the learning rate for mirostat. Lower learning rates result in slower adjustments, while higher learning rates make the algorithm more responsive. 0.1

Cited from Rockchip_RKLLM_SDK_CN.pdf

We use cookies to improve your browsing experience on our website, to analyse our website traffic, and to understand where our visitors are coming from.