[DCOS-38592] Scheduler DNS is not resolvable when the first tasks launch

This adds a check that the scheduler DNS entry is correct before completing the API server bring-up.

AFAICT this makes the scheduler resolve its own address right? Just wondering whether that guarantees in any way that other processes elsewhere in the cluster will be able to do the same yet... Or do we hope they will, by the time they start up?

@benclarkwood thanks! Looks like it is producing the desired effect, and we didn't get the failed tasks:
INFO  2018-06-28 09:15:08,629 [Thread-3] ApiServer:awaitSchedulerDns(64): Waiting for the scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory to resolve to <IP_ADDRESS>
WARN  2018-06-28 09:15:08,725 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
WARN  2018-06-28 09:15:13,725 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
WARN  2018-06-28 09:15:18,817 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
WARN  2018-06-28 09:15:23,817 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
WARN  2018-06-28 09:15:28,907 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
WARN  2018-06-28 09:15:33,907 [Thread-3] ApiServer:awaitSchedulerDns(98): Unable to look up scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory
INFO  2018-06-28 09:15:38,909 [Thread-3] ApiServer:awaitSchedulerDns(92): Resolved scheduler host kubernetes.marathon.autoip.dcos.thisdcos.directory to expected address <IP_ADDRESS>

In any case, I will run a couple more times to make extra sure!

Oops sorry for closing! I clicked the wrong button by mistake! 🤦♀

@bmcstdio glad it seems to be a fix. We had to do something similar in bootstrap awhile ago.
@porridge Yes, this relies on the task being able to self-resolve as a test for whether the task DNS is "live". I cannot remember how far up the chain it goes to get the answer to that question. I believe that it ultimately is querying up to DC/OS DNS on the masters (if the answer was already cached in the agent it wouldn't go upward) but I could be wrong. That's a big part of why it also checks that it's the right IP address to make sure it isn't just a record for an old copy of the service.
@jeid64 can you comment on the reliability of this method / speed of propagation? My rough knowledge of it is: navstar/spartan/et al slow to create DNS entry, very fast to propagate across cluster.
(FWIW, we've been successfully using this technique via the bootstrap utility in task bring up)

Autoip/containerip addresses are sourced from masters state.json. Every 30 seconds the masters state.json is polled and then the address is created, and then is pushed to all agents DNS resolvers very quickly. There is a chance that if dcos-dns is broken on an agent, that the scheduler on one agent will be able to resolve itself but tasks on another agent won't be able to, but that's pretty much out of our control.
VIP addresses are polled every 2 seconds by agents, so they're created a lot faster.

The main reason for waiting on the ApiServer to start is that it hosts the config artifact endpoints that are queried by the mesos fetcher as tasks are brought up.
So there are two dependencies before we should bring up tasks:

DNS for the scheduler needs to be resolvable at the tasks (previously VIP, now autoip)
scheduler itself needs to be running its API server

In practice the API server takes ~1s to actually load, but that's been enough of a delay to cause problems before


Sileo can't be automatic install when JB processing

Using the latest 2.0.11, but after JB finished, I'd try too many times, but Sileo always can't on screen, only after clicked refresh application or re-installing button, the Sileo can be on screen.
My mobile device is iPhone 13 ProMax, IOS 16.5

after about 1minute, see Sileo on screen, it seems the internet traffic issue.

after about 1minute, see Sileo on screen, it seems the internet traffic issue.


Error Compile ESP 8266 CORE 2.6.3

Hello, I'm having trouble compiling the example (sqlite3_sdfat) using ESP8266 on ArduinoIDE with core 2..6.3. someone with the same problem?

Hi, I just upgraded to 2.6.3 and tried.. I am able to compile and run it.
You probably may have to get the SdFat library which is causing this problem.
Be sure to get 1.0.7 of SdFat because the latest version causes trouble. See https://github.com/siara-cc/esp_arduino_sqlite3_lib/issues/5#issuecomment-487456337

Core ESP 2.6.3 integrated SDFAT into its core, it now uses the latest
version. I had problems here, it does not compile if used together with FS.
I think we need to work on SQLite to support the latest version of SDFAT.
Laercio Junior
Cel./Pessoal: (21) 99162-7440
Email Pessoal<EMAIL_ADDRESS>Skype: laercionit
Em dom., 8 de mar. de 2020 às 05:53, Arun<EMAIL_ADDRESS>escreveu:

Hi, I just upgraded to 2.6.3 and tried.. I am able to compile and run it.
You probably may have to get the SdFat library which is causing this
problem.
Be sure to get 1.0.7 of SdFat because the latest version causes trouble.
See #5 (comment)
https://github.com/siara-cc/esp_arduino_sqlite3_lib/issues/5#issuecomment-487456337
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/siara-cc/esp_arduino_sqlite3_lib/issues/13?email_source=notifications&email_token=AACPQHQTLM2GUEI7ZH7RQ6LRGNMJFA5CNFSM4K2THVQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOEQD5I#issuecomment-596181493,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACPQHSFMDTWVM27B5R3RBTRGNMJFANCNFSM4K2THVQQ
.

In this example I am using SPIFFS, which is more secure.
Using the ESP8266 Nodemcu 12-E with CORE 2.6.3
I am getting the errors below to compile.
I do not even know where to begin with. Today I use EDB as a database in SD, I want to replace it with SQLite in SD.

#include <stdio.h>
#include <stdlib.h>
#include <sqlite3.h>
#include <vfs.h>
#include <SPI.h>
#include <FS.h>
extern "C" {
#include "user_interface.h"
}
#include <ESP8266WiFi.h>

const char* data = "Callback function called";
static int callback(void *data, int argc, char **argv, char **azColName){
   int i;
   Serial.printf("%s: ", (const char*)data);
   for (i = 0; i<argc; i++){
       Serial.printf("%s = %s\n", azColName[i], argv[i] ? argv[i] : "NULL");
   }
   Serial.printf("\n");
   return 0;
}

int openDb(char *filename, sqlite3 **db) {
   int rc = sqlite3_open(filename, db);
   if (rc) {
       Serial.printf("Can't open database: %s\n", sqlite3_errmsg(*db));
       return rc;
   } else {
       Serial.printf("Opened database successfully\n");
   }
   return rc;
}

char *zErrMsg = 0;
int db_exec(sqlite3 *db, const char *sql) {
   Serial.println(sql);
   long start = micros();
   int rc = sqlite3_exec(db, sql, callback, (void*)data, &zErrMsg);
   if (rc != SQLITE_OK) {
       Serial.printf("SQL error: %s\n", zErrMsg);
       sqlite3_free(zErrMsg);
   } else {
       Serial.printf("Operation done successfully\n");
   }
   Serial.print(F("Time taken:"));
   Serial.println(micros()-start);
   return rc;
}

void setup() {
   Serial.begin(115200);
   sqlite3 *db1;
   char *zErrMsg = 0;
   int rc;
   char *sql;

   system_update_cpu_freq(SYS_CPU_160MHZ);

   SPI.begin();
   vfs_mount("/SD0", SS);

   sqlite3_initialize();

   // Open database 1
   if (openDb("/SD0/census2000names.db", &db1))  return;

   rc = db_exec(db1, "Select * from surnames where name = 'MICHELLE'");
   if (rc != SQLITE_OK) {
       sqlite3_close(db1);
       sqlite3_close(db2);
       return;
   }
   rc = db_exec(db1, "Select * from surnames where name = 'SPRINGER'");
   if (rc != SQLITE_OK) {
       sqlite3_close(db1);
       sqlite3_close(db2);
       return;
   }

   sqlite3_close(db1);

}

void loop() {
}

Like I said, I think temporarily you can get it working with SdFat 1.0.7 downloaded using Library Manager till I fix it?

@siara-cc
Unfortunately this is not possible.
In CORE 2.6.3 of ESP8266 (Arduino) he incorporated SDFAT, this is in the core of esp8266, there is no way to downgrade.
So I looked for help to try to put SQLite to run on this core, it would be a great replacement for EDB (SLOW)

Is this issue resolved, as i am still getting below errors
C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:6:8: error: 'SdFat' does not name a type     6 | static SdFat *sdFats[] = {NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL};       |        ^~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'vfs_vol* sdfat_mount(const char*, int)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:15:5: error: 'SdFat' was not declared in this scope; did you mean 'sdfat::SdFat'?    15 |     SdFat *sdFat = new SdFat();       |     ^~~~~       |     sdfat::SdFat In file included from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:402:17: note: 'sdfat::SdFat' declared here   402 | typedef SdFat32 SdFat;       |                 ^~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:15:12: error: 'sdFat' was not declared in this scope; did you mean 'sdfat'?    15 |     SdFat *sdFat = new SdFat();       |            ^~~~~       |            sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:15:24: error: expected type-specifier before 'SdFat'    15 |     SdFat *sdFat = new SdFat();       |                        ^~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:18:5: error: 'sdFats' was not declared in this scope; did you mean 'sdfat'?    18 |     sdFats[sd_idx] = sdFat;       |     ^~~~~~       |     sdfat In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'vfs_file* sdfat_open(const char*, const char*)': c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:122:19: error: 'O_RDONLY' was not declared in this scope; did you mean 'sdfat::O_RDONLY'?   122 | #define O_READ    O_RDONLY       |                   ^~~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:36:18: note: in expansion of macro 'O_READ'    36 |         imode |= O_READ;       |                  ^~~~~~ c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:76:15: note: 'sdfat::O_RDONLY' declared here    76 | const uint8_t O_RDONLY = O_READ;       |               ^~~~~~~~ c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:123:19: error: 'O_WRONLY' was not declared in this scope; did you mean 'sdfat::O_WRONLY'?   123 | #define O_WRITE   O_WRONLY       |                   ^~~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:38:21: note: in expansion of macro 'O_WRITE'    38 |            imode |= O_WRITE;       |                     ^~~~~~~ c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:80:15: note: 'sdfat::O_WRONLY' declared here    80 | const uint8_t O_WRONLY = O_WRITE;       |               ^~~~~~~~ c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:123:19: error: 'O_WRONLY' was not declared in this scope; did you mean 'sdfat::O_WRONLY'?   123 | #define O_WRITE   O_WRONLY       |                   ^~~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:41:18: note: in expansion of macro 'O_WRITE'    41 |         imode |= O_WRITE;       |                  ^~~~~~~ c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:80:15: note: 'sdfat::O_WRONLY' declared here    80 | const uint8_t O_WRONLY = O_WRITE;       |               ^~~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:43:22: error: 'O_CREAT' was not declared in this scope; did you mean 'sdfat::O_CREAT'?    43 |             imode |= O_CREAT;       |                      ^~~~~~~       |                      sdfat::O_CREAT In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:94:15: note: 'sdfat::O_CREAT' declared here    94 | const uint8_t O_CREAT = 0X40;       |               ^~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:45:22: error: 'O_TRUNC' was not declared in this scope; did you mean 'sdfat::O_TRUNC'?    45 |             imode |= O_TRUNC;       |                      ^~~~~~~       |                      sdfat::O_TRUNC In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:90:15: note: 'sdfat::O_TRUNC' declared here    90 | const uint8_t O_TRUNC = 0X10;       |               ^~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:48:18: error: 'O_AT_END' was not declared in this scope; did you mean 'sdfat::O_AT_END'?    48 |         imode |= O_AT_END;       |                  ^~~~~~~~       |                  sdfat::O_AT_END In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:92:15: note: 'sdfat::O_AT_END' declared here    92 | const uint8_t O_AT_END = 0X20;       |               ^~~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:49:18: error: 'O_CREAT' was not declared in this scope; did you mean 'sdfat::O_CREAT'?    49 |         imode |= O_CREAT;       |                  ^~~~~~~       |                  sdfat::O_CREAT In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:94:15: note: 'sdfat::O_CREAT' declared here    94 | const uint8_t O_CREAT = 0X40;       |               ^~~~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:51:17: error: 'sdFats' was not declared in this scope; did you mean 'sdfat'?    51 |     File file = sdFats[sd_idx]->open(name + 5, imode);       |                 ^~~~~~       |                 sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_remove(const char*)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:63:5: error: 'SdFat' was not declared in this scope; did you mean 'sdfat::SdFat'?    63 |     SdFat *sdFat = sdFats[sd_idx];       |     ^~~~~       |     sdfat::SdFat In file included from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:402:17: note: 'sdfat::SdFat' declared here   402 | typedef SdFat32 SdFat;       |                 ^~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:63:12: error: 'sdFat' was not declared in this scope; did you mean 'sdfat'?    63 |     SdFat *sdFat = sdFats[sd_idx];       |            ^~~~~       |            sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:63:20: error: 'sdFats' was not declared in this scope; did you mean 'sdfat'?    63 |     SdFat *sdFat = sdFats[sd_idx];       |                    ^~~~~~       |                    sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_exists(const char*)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:73:5: error: 'SdFat' was not declared in this scope; did you mean 'sdfat::SdFat'?    73 |     SdFat *sdFat = sdFats[sd_idx];       |     ^~~~~       |     sdfat::SdFat In file included from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:402:17: note: 'sdfat::SdFat' declared here   402 | typedef SdFat32 SdFat;       |                 ^~~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:73:12: error: 'sdFat' was not declared in this scope; did you mean 'sdfat'?    73 |     SdFat *sdFat = sdFats[sd_idx];       |            ^~~~~       |            sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:73:20: error: 'sdFats' was not declared in this scope; did you mean 'sdfat'?    73 |     SdFat *sdFat = sdFats[sd_idx];       |                    ^~~~~~       |                    sdfat C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_read(vfs_file*, void*, size_t)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:89:49: error: invalid conversion from 'void*' to 'uint8_t*' {aka 'unsigned char*'} [-fpermissive]    89 |     sint32_t ret = ((File *)fd->file_obj)->read(ptr, len);       |                                                 ^~~       |                                                 |       |                                                 void* In file included from c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:58,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:70:23: note:   initializing argument 1 of 'virtual int fs::File::read(uint8_t*, size_t)'    70 |     int read(uint8_t* buf, size_t size) override;       |              ~~~~~~~~~^~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_write(vfs_file*, const void*, size_t)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:96:58: error: no matching function for call to 'fs::File::write(const void*&, size_t&)'    96 |     sint32_t ret = ((File *)fd->file_obj)->write(ptr, len);       |                                                          ^ In file included from c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:58,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:59:12: note: candidate: 'virtual size_t fs::File::write(const uint8_t*, size_t)' (near match)    59 |     size_t write(const uint8_t *buf, size_t size) override;       |            ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:59:12: note:   conversion of argument 1 would be ill-formed: C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:96:50: error: invalid conversion from 'const void*' to 'const uint8_t*' {aka 'const unsigned char*'} [-fpermissive]    96 |     sint32_t ret = ((File *)fd->file_obj)->write(ptr, len);       |                                                  ^~~       |                                                  |       |                                                  const void* In file included from c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\common\fsapiconstants.h:58,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatFile.h:36,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatVolume.h:28,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/ExFatLib/ExFatLib.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:33,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:58:12: note: candidate: 'virtual size_t fs::File::write(uint8_t)'    58 |     size_t write(uint8_t) override;       |            ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:58:12: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:89:33: note: candidate: 'template<class T> size_t fs::File::write(T&)'    89 |     template<typename T> size_t write(T &src){       |                                 ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/FS.h:89:33: note:   template argument deduction/substitution failed: C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:96:58: note:   candidate expects 1 argument, 2 provided    96 |     sint32_t ret = ((File *)fd->file_obj)->write(ptr, len);       |                                                          ^ In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Stream.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/HardwareSerial.h:32,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Arduino.h:288,                  from c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\sdfatconfig.h:46,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/common/SysCall.h:33,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:31,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:76:23: note: candidate: 'size_t Print::write(int8_t)'    76 |         inline size_t write(int8_t c) { return write((uint8_t) c); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:76:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:75:23: note: candidate: 'size_t Print::write(char)'    75 |         inline size_t write(char c) { return write((uint8_t) c); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:75:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:73:23: note: candidate: 'size_t Print::write(long long unsigned int)'    73 |         inline size_t write(unsigned long long t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:73:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:72:23: note: candidate: 'size_t Print::write(long long int)'    72 |         inline size_t write(long long t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:72:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:71:23: note: candidate: 'size_t Print::write(long unsigned int)'    71 |         inline size_t write(unsigned long t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:71:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:70:23: note: candidate: 'size_t Print::write(long int)'    70 |         inline size_t write(long t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:70:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:69:23: note: candidate: 'size_t Print::write(unsigned int)'    69 |         inline size_t write(unsigned int t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:69:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:68:23: note: candidate: 'size_t Print::write(int)'    68 |         inline size_t write(int t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:68:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:67:23: note: candidate: 'size_t Print::write(short unsigned int)'    67 |         inline size_t write(unsigned short t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:67:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:66:23: note: candidate: 'size_t Print::write(short int)'    66 |         inline size_t write(short t) { return write((uint8_t)t); }       |                       ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:66:23: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:62:16: note: candidate: 'size_t Print::write(const char*, size_t)' (near match)    62 |         size_t write(const char *buffer, size_t size) {       |                ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:62:16: note:   conversion of argument 1 would be ill-formed: C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:96:50: error: invalid conversion from 'const void*' to 'const char*' [-fpermissive]    96 |     sint32_t ret = ((File *)fd->file_obj)->write(ptr, len);       |                                                  ^~~       |                                                  |       |                                                  const void* In file included from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Stream.h:27,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/HardwareSerial.h:32,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Arduino.h:288,                  from c:\users\pushe\onedrive\documents\arduinodata\packages\esp8266\hardware\esp8266\3.0.0\libraries\esp8266sdfat\src\sdfatconfig.h:46,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/common/SysCall.h:33,                  from C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\libraries\ESP8266SdFat\src/SdFat.h:31,                  from C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:3: C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:56:16: note: candidate: 'size_t Print::write(const char*)'    56 |         size_t write(const char *str) {       |                ^~~~~ C:\Users\pushE\OneDrive\Documents\ArduinoData\packages\esp8266\hardware\esp8266\3.0.0\cores\esp8266/Print.h:56:16: note:   candidate expects 1 argument, 2 provided C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_lseek(vfs_file*, sint32_t, int)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:105:38: error: 'class fs::File' has no member named 'seekSet'; did you mean 'seek'?   105 |         if ( ((File *)fd->file_obj)->seekSet(off) )       |                                      ^~~~~~~       |                                      seek C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:108:38: error: 'class fs::File' has no member named 'seekCur'; did you mean 'seek'?   108 |         if ( ((File *)fd->file_obj)->seekCur(off) )       |                                      ^~~~~~~       |                                      seek C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:111:38: error: 'class fs::File' has no member named 'seekEnd'; did you mean 'seek'?   111 |         if ( ((File *)fd->file_obj)->seekEnd(off) )       |                                      ^~~~~~~       |                                      seek C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'sint32_t sdfat_flush(vfs_file*)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:118:27: error: 'class fs::File' has no member named 'sync'   118 |   ((File *)fd->file_obj)->sync();       |                           ^~~~ C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp: In function 'uint32_t sdfat_size(vfs_file*)': C:\Users\pushE\OneDrive\Documents\Arduino\libraries\Sqlite3_for_ESP8266\src\sdfat_fns.cpp:123:34: error: 'class fs::File' has no member named 'fileSize'   123 |   return ((File *)fd->file_obj)->fileSize();       |                                  ^~~~~~~~ exit status 1 Error compiling for board Generic ESP8266 Module. Board at COM5 is not available

I think SdFat is not installed.  Please download and installed the library v1.0.7 from https://github.com/greiman/SdFat and try compiling again

@siara-cc  is sdfat 2.2.2 ( latest ) version not supported ? Do we have any plans to support that ?

@bhullar89 There was some issue quite some time back with a later version. I don't remember what the issue was.  If that issue has been fixed with sdfat 2.2.2, it may work. If you find it working please let me know so I can remove the note on restriction :-)

I think SdFat is not installed. Please download and installed the library v1.0.7 from https://github.com/greiman/SdFat and try compiling again

i had the same problem and install SdFat v 1.0.7 and My problem Solved.
by the way for those wanna use inside flash memory insted of  sdcard  This will confusing
its be good if seprate sd users from flash users.
thanks.


There are bugs when using Chinese

+---------+-----------------+
| 键     | 值             |
+---------+-----------------+
| V版本 | V 0.2.2 0dafdb4 |
+---------+-----------------+

Yeah, this is already a known issue (#27) because the implementation is blocked by a missing feature in V itself.
The width of CJK letters is not calculated correctly.
Recently someone made progress there, but I lack free time right now to check if it works for termtable.
Feel free to make a PR on your own or you might have to wait a few weeks :(

Closing as duplicate of #27

@xiusin I have implemented support for chinese. It was actually a lot easier and faster than I thought.
I would love, if you could give it a try and give feedback :heart:


Fix the coding style issue

Summary
Please ignore the below false alarm:
/home/xiaoxiang/backup/os/nuttx/apps/examples/camera/camera_main.c:639:44: error: Operator/assignment must be followed with whitespace
/home/xiaoxiang/backup/os/nuttx/apps/interpreters/quickjs/qjsmini.c:105:23: error: Mixed case identifier found
/home/xiaoxiang/backup/os/nuttx/apps/system/vi/vi.c:540:57: error: Multiple data definitions

Impact
Minor change
Testing
Pass CI

What about this error?
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/testing/mtd_config_fs/mtd_config_fs_test_main.c:1473:1: error: Blank line contains

What about this error?
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/testing/mtd_config_fs/mtd_config_fs_test_main.c:1473:1: error: Blank line contains

Done.


Update docs with more cautions about beats compatability

Please update the docs to reflect beyond wire compatibility, using beats beyond 7.10.x risks beats using logic that may not be compatible with the opensearch backend.
Example - beats often make use of ingest pipelines - and newer ingest pipelines may use features that require the opensearch backend be compatible with > 7.10.x

@ryn9 is the logic you are referring to in the open source beats?

@elfisher - it is applicable to both opensource and non-opensource versions

Hi @ryn9, I added a note to this page: https://opensearch.org/docs/clients/agents-and-ingestion-tools/index/#downloads
Is your thinking here that we should just recommend 7.10.2 across the board?

I would recommend at least a caution that use of beats > 7.10 risks potential compatibility issues with use of ingest pipelines.
Elastic has pushed a fair amount of ETL into the ingest pipelines that lives inside of Elasticsearch, instead of ETL being done in beats, logstash, or other part of the data ingest process.  As such - there very well could be ingest pipeline code not compatible with Opensearch. Again - this is applicable to both opensource and non-opensource versions of beats.  Hopefully I am being clear - but do let me know if you need more info.


Add Chuvash

Please send your pull requests the develop branch.
Don't forget to add the change to CHANGELOG.md.

Note: Sometimes the development moves very fast. It is highly
recommended that you update your branch of develop before creating a
pull request to send us your changes. This makes everyone's lives
easier (including yours) and helps us out on the development team.
Thanks!

Does the pull request solve a related issue?
If so, can you reference the issue?
What does the pull request accomplish? Use a list if needed.
If it includes major visual changes please add screenshots.

Please base your PR against the develop branch so we can see the actual changes :-)

I'll create a new Pull Request to the branch.


Cannot merge several Noto fonts for historic languages

I ran pyftmerge to merge the following six Noto fonts with other Noto historic fonts (in https://github.com/googlei18n/noto-fonts/tree/master/unhinted) and got some errors.
NotoSansGothic-Regular.ttf
NotoSansOsmanya-Regular.ttf
NotoSansPhoenician-Regular.ttf
NotoSansBamum-Regular.ttf
NotoSansShavian-Regular.ttf
NotoSansCuneiform-Regular.ttf
For example, you can run the following commands to reproduce the problem.
$ pyftmerge     NotoSansGothic-Regular.ttf NotoSansOgham-Regular.ttf
$ pyftmerge     NotoSansInscriptionalPahlavi-Regular.ttf NotoSansOsmanya-Regular.ttf
$ pyftmerge NotoSansAvestan-Regular.ttf NotoSansPhoenician-Regular.ttf
$ pyftmerge     NotoSansBamum-Regular.ttf NotoSansRunic-Regular.ttf
$ pyftmerge NotoSansSamaritan-Regular.ttf NotoSansShavian-Regular.ttf
$ pyftmerge NotoSansCoptic-Regular.ttf NotoSansCuneiform-Regular.ttf
An example error message I got is:
Traceback (most recent call last):
File "/Users/xiangye/.venv/bin/pyftmerge", line 11, in
sys.exit(main())
File "/Users/xiangye/.venv/lib/python3.5/site-packages/FontTools/fontTools/misc/loggingTools.py", line 372, in wrapper
return func(_args, *_kwds)
File "/Users/xiangye/.venv/lib/python3.5/site-packages/FontTools/fontTools/merge.py", line 943, in main
font = merger.merge(args)
File "/Users/xiangye/.venv/lib/python3.5/site-packages/FontTools/fontTools/merge.py", line 813, in merge
table = clazz(tag).merge(self, tables)
File "/Users/xiangye/.venv/lib/python3.5/site-packages/FontTools/fontTools/merge.py", line 508, in merge
assert (table is not None and table is not NotImplemented), "Have duplicates to resolve for font %d but no GSUB" % (i + 1)
AssertionError: Have duplicates to resolve for font 2 but no GSUB
wondering whether the fonts or pyftmerge have issues.

The issue is triggered by the second font file in your command line, which does not contain GSUB table but has some duplicated glyphs with the first font file.
Specifically followiong fonts are not recommended to be put as the second parameter since they do not contain GSUB table,
NotoSansOgham-Regular.ttf
NotoSansOsmanya-Regular.ttf
NotoSansPhoenician-Regular.ttf
NotoSansRunic-Regular.ttf
NotoSansShavian-Regular.ttf
NotoSansCuneiform-Regular.ttf
This problem can be solved by exchange the parameter order. Instead of pyftmerge A.ttf B.ttf, use pyftmerge B.ttf A.ttf
However, it is a bug in pyftmerge script I think, which needs to dig deeper to solve.

Thanks Bo.
When merging Noto, I suggest you keep NotoSansLGC as the first font in the list.  When hitting this problem, there are a few options to resolve it:

Add GSUB for font without it.  We need to identify the script tag to use. AND it is only safe to do so if we know there is no script-specific shaping logic for that script.  In general, it's easiest to avoid this approach.

Avoid needing a GSUB. We need GSUB when a character has glyphs in multiple fonts.  Most probably this is only relevant for space, period, comma, etc glyphs in these fonts.  If we can satisfy that the duplicate glyphs are identical or close, then we don't need to record a substitution.

I suggest you do the latter.  You can start by drawing the glyph to a recording pen font.getGlyphSet()[glyphname].draw(pen) and comparing the recordings as well as pen.width.  Something like that.
Or first, just identify which characters are involved. Also, I'm open to making this a warning initially.  We did that for another issue.  See 18b905b06d436ed961fcabe85001a668679d46f5

Thanks for the advice.
I tried your second suggestions and made this a warning and following are the duplicated glyphs the script found in merging. As you predicted, all of them are space, period, comma and simple symbols that are closed to each other. I think we can tolerant the difference and ignore these warnings. I'll submit a pull request to change it to a warning.
"When merging Noto, I suggest you keep NotoSansLGC as the first font in the list."

Can I assume this is because we want to make NotoSansLGC as the base font file, and append glyphs in other font files to it? By doing so, the indices of the glyphs in LGC (which are used frequently) is small. May I know what's the benefit of doing this? I guess it makes glyph searching in the font file faster?

"We need GSUB when a character has glyphs in multiple fonts"

It seems we do not overwrite the duplicated glyph with the new glyph directly but use GSUB to substitute it. Is it because overwriting the duplicated glyph in glyf table costs too much?

"I suggest you do the latter. You can start by drawing the glyph to a recording pen font.getGlyphSet()[glyphname].draw(pen) and comparing the recordings as well as pen.width. Something like that."

Yeah, I believe it's a powerful comparing tool but I don't quite know how to use it. Alternatively, I built two webpages showing all unicode characters in a font to visually see the difference between characters in two fonts. Plus, I found this webpage is very helpful for viewing a font: http://bluejamesbond.github.io/CharacterMap/

Following are duplciated glyphs:
'uni0000' control
'uni000D‘ control
'uni00A0' control
'uni3001' 、
'uni3002' 。
'uni3008' 〈
'uni3009' 〉
'uni300A' 《
'uni300B'  》
'uni300C' 「
'uni300D'  」
'uni300E' 『
'uni300F’ 』
'uni3010' 【
'uni3011'  】
'uni3014' 〔
'uni3015'〕
'uni3016'〖
'uni3017'〗
'uni3018' 〘
'uni3019'〙
'uni301A' 〚
'uni301B' 〛
'uniFEFF' ZERO WIDTH NO-BREAK SPACE
'space'

Read the fonttools merge.py source code. Got some answers for previous posted questions, but still has some questions which are written in bold.

"When merging Noto, I suggest you keep NotoSansLGC as the first font in the list."

Some tables are using 'first' as their merging logic, which keeps the table in the first font and discard those in other fonts, such as prep, fpgm, cvt.
Some keys in tables are also using 'first' as their merging logic, like 'names' in name table, etc.
From my understanding, always using NotoSansLGC as the first parameter keeps the information in LGC font as much as possible in the result. This is because NotoSansLGC font is more authentic and reliable.(?)

"We need GSUB when a character has glyphs in multiple fonts"

A glyph can be selected by the combination of codepoint and language tag. For example, <uni00A0, CN> may selects a different glyph than <uni00A0, JP>, though their codepoint is the same. Using GSUB table to substitute the glyph based on the language tag. However, we have to drop extra duplicate glyphs in different fonts if their langSys tag are the same.(?)

we have to drop extra duplicate glyphs in different fonts if their langSys tag are the same

Well, what can a poor little merging tool possibly do when the user wants to merge two fonts where the same glyph looks different in the same language? But does it actually happen in practice with Noto? For which glyphs in what languagesystems?

But does it actually happen in practice with Noto?

Yes.

For which glyphs in what languagesystems?

Unfortunately, following fonts that will be mentioned does not contain GSUB thus does not have a language system tag..
'space' and 'uni00A0' are different in NotoSans-Regular and NotoSansEgyptianHieroglyphs-Regular with 532 and 500 advance width respectively.
'000D' in NotoSans-Regular and NotoSansLinearB-Regular. Advance width are 1044 and 532 respectively.
'02cd' in NotoSans-Regular and NotoSansLisu-Regular  uni02CD#0:(GlyphCoordinates([(25, -156),(633, -156),(633, -301),(25, -301)]), [3], array('B', [1, 1, 1, 1])), uni02CD#1:(GlyphCoordinates([(150, 0),(150, 175),(818, 175),(818, 0)]), [3], array('B', [1, 1, 1, 1]))
The rectangle shape is similar but one is longer and the other is shorter.
'3018' in NotoSans-Regular and NotoSansYi-Regular. uni3018#0:(GlyphCoordinates([(1050, -100),(986, -200),(550, 45),(550, 1213),(986, 1458),(1050, 1358),(912, 1279),(912, -21),(670, 111),(792, 48),(792, 1210),(670, 1147)]), [7, 11], array('B', [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])), uni3018#1:(GlyphCoordinates([(1050, -100),(986, -200),(550, 45),(550, 1217),(986, 1462),(1050, 1362),(912, 1283),(912, -21),(670, 111),(792, 48),(792, 1214),(670, 1151)]), [7, 11], array('B', [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]))
The shapes are very very similar to each other except for a tiny offset of two or three control points.
These are just part of the duplicates glyphs with different shapes, full list is posted in a previous post in this thread.
From my understanding, we should make script table in GSUB required, because the script tag helps us better handle the duplicates in font merging.

Well, what can a poor little merging tool possibly do when the user wants to merge two fonts where the same glyph looks different in the same language?

From the description above, we know that all the duplicates in our Noto are symbols and punctuation, with slightly different shapes. It should be OK to keep the first glyph and drops the rest. However, our little merging tool could be smarter to handle this case.
For example, if font A and font B both contain glyph comma ',' with slightly different shapes, and font A and B do not contain script tag that we can not fetch the correct glyph by language tag, we can still fetch the promising glyph by its context. Fetch the ',' from font B if ',' appears adjacent to some other characters in B, otherwise fetch ',' from font A.
Ch and En both contains period with different shapes, using '.' when it's next to another English letter while using '。' when it's next to a Chinese character。
Probably it involves contextual substitution concept.
Just my $.02


Fix for HTTP_RAW_POST_DATA on newer PHP; it is often not populated.

PHP 5.6 complains about deprecation if $HTTP_RAW_POST_DATA is used. Fix to use php://input if $HTTP_RAW_POST_DATA is unset. If $HTTP_RAW_POST_DATA is set, the old behavior is used, so this should have no impact if it's set. If unset, the current code would be broken as $HTTP_RAW_POST_DATA  is not defined, and now it will work. Thanks!

CI2 is in security maintenance mode until October and no longer receives other bugfixes, sorry.
https://github.com/bcit-ci/CodeIgniter/blob/2.2-stable/README.md


Links directly to 27 PDF files under http://www.open-bio.org/w/

e.g. http://www.open-bio.org/w/images/2/25/Brigham_Circle_-_Countway_Map.pdf from Codefest_2010.mediawiki aka https://github.com/OBF/wiki/blob/main/Codefest_2010.md
$ grep "http://www.open-bio.org/w/" *.mediawiki  | wc -l
27

We don't seem to have this file via the XML dump - but why not?
Since the old site is live could always fetch over HTTP(s) and add to the repository. See also #6 and if we should use a sub-directory.

$ grep -oh "http://www.open-bio.org/w/.*\.pdf" *.mediawiki
http://www.open-bio.org/w/images/b/b6/BOSC2008_program.pdf
http://www.open-bio.org/w/images/b/b6/BOSC2008_program.pdf
http://www.open-bio.org/w/images/c/c7/Menager_Mobyle_abstract.pdf
http://www.open-bio.org/w/images/e/e5/Smith_R_Intermine_abstract.pdf
http://www.open-bio.org/w/images/6/69/Gordon_SemanticWeb_abstract.pdf
http://www.open-bio.org/w/images/3/3c/Rice_Soaplab2_abstract.pdf
http://www.open-bio.org/w/images/2/27/Sanders_Arcoleo_BeeSpace_abstract.pdf
http://www.open-bio.org/w/images/6/6c/Jenkinson_BioDas_abstract.pdf
http://www.open-bio.org/w/images/2/20/Arakawa_E-Cell3D_abstract.pdf Abstract] [http://www.g-language.org/data/gaou/presentation/BOSC08-ECell3D.pdf
http://www.open-bio.org/w/images/c/ca/Rice_EMBOSS_abstract.pdf
http://www.open-bio.org/w/images/1/10/Heuer_BioJava_abstract.pdf
http://www.open-bio.org/w/images/a/a9/Wilson_Make_abstract.pdf
http://www.open-bio.org/w/images/6/63/OConnor_SolexaTools_abstract.pdf
http://www.open-bio.org/w/images/d/da/Wilkinson_BioMoby_abstract.pdf
http://www.open-bio.org/w/images/5/59/Orvis_Ergatis_abstract.pdf
http://www.open-bio.org/w/images/7/7e/Kallio_Chipster_abstract.pdf
http://www.open-bio.org/w/images/7/75/Bhagat_myExperiment_abstract.pdf
http://www.open-bio.org/w/images/2/22/Smith_T_BioHDF_Abstract_revised.pdf
http://www.open-bio.org/w/images/f/fb/Kitzmiller_OpenHeliSphere_abstract.pdf
http://www.open-bio.org/w/images/3/3a/Lee_Apollo_abstract.pdf
http://www.open-bio.org/w/images/3/3b/Lapp_BioSQL_abstract.pdf
http://www.open-bio.org/w/images/1/10/Haider_EMBRACE_abstract.pdf
http://www.open-bio.org/w/images/e/e3/Antao_Biopython_abstract.pdf
http://www.open-bio.org/w/images/d/dd/Bosc2014-bioinform.pdf
http://www.open-bio.org/w/images/2/2c/BOSC2015-schedule.pdf
http://www.open-bio.org/w/images/8/85/BOSC_2010_Cloud_BioLinux.pdf
http://www.open-bio.org/w/images/2/25/Brigham_Circle_-_Countway_Map.pdf


[Bug]: Remove the OK button from the management database permissions interface

ODC version
ODC 4.2.4
OB version

What happened?
Remove the OK button from the management library permissions interface.

What did you expect to happen?

How can we reproduce it (as minimally and precisely as possible)?
Enter the management database permissions interface.
Anything else we need to know?
No response
Cloud
No response

Fixed In Commit f35f340b944f118de92e7563716f08feb1f1bf37
------
咏和
Tue Mar 5 09:33:13 2024 +0800

pass


Inferring on any instance in the kitti-raw dataset

Thank you for your great work. I want to infer on any instance in the kitti-raw dataset, but the structure of kitti-raw and KITTI-DC is not consistent. That is to say, given a frame of raw image, raw lidar data and the relationship parameters between the two, can I directly obtain the completed result with the pretrained model? Can you give me some help, such as a script? thank you!

{
"rgb": "rawdata/2011_09_30/2011_09_30_drive_0028_sync/image_03/data/0000001643.png",
"depth": "data_depth_velodyne/train/2011_09_30_drive_0028_sync/proj_depth/velodyne_raw/image_03/0000001643.png",
"gt": "data_depth_annotated/train/2011_09_30_drive_0028_sync/proj_depth/groundtruth/image_03/0000001643.png",
"K": "rawdata/2011_09_30/calib_cam_to_cam.txt"
}

非常感谢您的回复！您的模型是在KITTI_DC数据集上训练和测试的，这个数据集提供了列表中的depth和gt。我现在想在KITTI上的其它数据集上使用您的预训练模型来进行推理，比如KITTI Odometry数据集，这个数据集只包含了rgb图像、原始点云数据以及一些转换参数。所以我想实现输入rgb图像和原始点云，将点云投影到图像平面上获取深度，然后直接接入您的预训练模型上进行推理，请问您可以帮忙吗？（因为我没有找到KITTI_DC中depth和gt的生成方式）非常感谢！

Hi,
Okay, I got your points. To project the 3D point cloud into depth image, maybe you can infer to this. I think it would help.
Best,

Thanks a bunch!


DDCYLS-5144: Update to play 3.0

you have akka in your conf/logback.xml file

scalafmt can be upgraded