HOW TO FIX - Traceback (most recent call last):   File "/usr/local/bin/twint", line 33, in   && File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/token.py", line 69, in refresh raise RefreshTokenException('Could not find the Guest token in HTML') twint.token.RefreshTokenException: Could not find the Guest token in HTML

I really love this app and I have only used it once. But now I want to use it and I can't. I have searched so much that my eyes are sore, why am I getting these errors and why can't I find a fix for it!! hehe...
Linux Kali-ROG 5.14.0-kali4-amd64 #1 SMP Debian 5.14.16-1kali1 (2021-11-05) x86_64 GNU/Linux
Please help not just me but anyone who searches the 400 pages of information to find an answer.  9 hours of work to try this, the time I will never get back and I just want to chuck in Linux altogether and do anything else.

[] Python version is 3.6;
[] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;

These are the steps I took to install twint and get try and get it to work on 2 versions of Linux and 1 Deb.  I have spent 9 hours trying my best, but no fix anywhere, none at all, not one.
What errors am I getting, all listed below
1: Install on Kali. This did not work for me.
Linux Kali-ROG 5.14.0-kali4-amd64 #1 SMP Debian 5.14.16-1kali1 (2021-11-05) x86_64 GNU/Linux
Instructions for install : https://github.com/twintproject/twint
    git clone –depth=1 https://github.com/twintproject/twint.git
    cd twint
    pip3 install . -r requirements.txt

Try and run these commands to see if it works.
   $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

2: Install on Kali AWS. This did not work for me.
Linux kali 5.15.0-kali2-cloud-amd64 #1 SMP Debian 5.15.5-2kali2 (2021-12-22) x86_64 GNU/Linux
It seems that twint and AWS do not work due to
Instructions for install : https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
    sudo git clone –depth=1 https://github.com/twintproject/twint.git
    sudo cd twint
    sudo pip3 install . -r requirements.txt
    sudo pip3 install twint

Try and run these commands to see if it works.
    $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

3: Install on Google Cloud Shell and This worked for me.
Lightbox Linux cs-899333161534-default-boost-h8jtm 5.10.68+ #1 SMP Wed Dec 1 10:07:21 UTC 2021 x86_64 GNU/Linux
Instructions for install: https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
FYI 1 - Google didn't like the command git clone --depth=1
So I removed it and it  installed.
FY 2 - Using pip3 install twint command said all was installed, but I added sudo
$ sudo git clone https://github.com/twintproject/twint.git
$ ls
$ cd twint
$ ls
$ sudo pip3 install . -r requirements.txt
$ sudo pip3 install twint
Ran commands to see if it works.
    $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

I was able to run twint until I logged out and logged back in, it didn't work anymore.
The Kali twint install journey.
My Install on Kali Linux.
┌──(kali㉿kali)-[~]
└─$ git clone --depth=1 https://github.com/twintproject/twint.git
Cloning into 'twint'...
remote: Enumerating objects: 47, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (44/44), done.
remote: Total 47 (delta 3), reused 14 (delta 0), pack-reused 0
Receiving objects: 100% (47/47), 42.95 KiB | 1.59 MiB/s, done.
Resolving deltas: 100% (3/3), done.
┌──(kali㉿kali)-[~]
└─$ ls
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos  bash.txt  nmap  twint
┌──(kali㉿kali)-[~]
└─$ cd twint
┌──(kali㉿kali)-[~/twint]
└─$cd twint
┌──(kali㉿kali)-[~/twint]
└─$ pip3 install . -r requirements.txt
Command 'pip3' not found, but can be installed with:
sudo apt install python3-pip
Do you want to install it? (N/y)y
sudo apt install python3-pip
┌──(kali㉿kali)-[~/twint]
└─$ pip3 install . -r requirements.txt
WARNING: The script twint is installed in '/home/kali/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed aiohttp-socks-0.4.1 cchardet-2.1.7 dataclasses-0.6 elasticsearch-7.16.2 fake-useragent-0.1.11 geographiclib-1.52 geopy-2.2.0 googletransx-2.4.2 pandas-1.3.5 schedule-1.1.0 twint-2.1.21
(I added two commands for the path later on in the install, see far below).
┌──(kali㉿kali)-[~/.local]
└─$ cd bin
┌──(kali㉿kali)-[~/.local/bin]
└─$ ls
cchardetect  translate  twint
┌──(kali㉿kali)-[~/.local/bin]
└─$ twint -u networkchuck
Traceback (most recent call last):
File "/usr/local/bin/twint", line 33, in
sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')())
File "/usr/local/bin/twint", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in
from . import run
File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in
from . import datelock, feed, get, output, verbose, storage
File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in
from aiohttp_socks import ProxyConnector, ProxyType
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in
from .connector import (
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in
from aiohttp.helpers import CeilTimeout  # noqa
ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
┌──(kali㉿kali)-[~/.local/bin]
└─$ echo $SHELL
/bin/zsh
┌──(kali㉿kali)-[~/.local/bin]
└─$ bash
┌──(kali㉿kali)-[~/.local/bin]
└─$ myip
Command 'myip' not found, did you mean:
command 'mzip' from deb mtools
Try: sudo apt install
┌──(kali㉿kali)-[~/.local/bin]
└─$ twint -u networkchuck -s "raspberry pi"
Traceback (most recent call last):
File "/usr/local/bin/twint", line 33, in
sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')())
File "/usr/local/bin/twint", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in
from . import run
File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in
from . import datelock, feed, get, output, verbose, storage
File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in
from aiohttp_socks import ProxyConnector, ProxyType
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in
from .connector import (
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in
from aiohttp.helpers import CeilTimeout  # noqa
ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
──(kali㉿kali)-[~/.local/bin]
└─$ cd ..
┌──(kali㉿kali)-[~/.local]
└─$ cd..
cd..: command not found
┌──(kali㉿kali)-[~/.local]
└─$ cd ..
┌──(kali㉿kali)-[~]
└─$
┌──(kali㉿kali)-[~]
└─$ twint -u networkchuck -s "raspberry pi"
Command 'twint' not found, did you mean:
command 'twine' from deb twine
Try: sudo apt install
┌──(kali㉿kali)-[~/twint]
└─$ python3
Python 3.9.9 (main, Dec 16 2021, 23:13:29)
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

┌──(kali㉿kali)-[~/twint]
└─$ twint -u networkchuck -s "raspberry pi"
Command 'twint' not found, did you mean:
command 'twine' from deb twine
Try: sudo apt install
┌──(kali㉿kali)-[~]
└─$ sudo apt update && sudo apt upgrade
┌──(kali㉿kali)-[~]
└─$ systemctl reboot -i
┌──(kali㉿kali)-[~/twint]
└─$cd twint
┌──(kali㉿kali)-[~/twint]
└─$ chmod +x setup.py
┌──(kali㉿kali)-[~/twint]
└─$ python3 setup.py
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: no commands supplied
──(kali㉿kali)-[~/twint]
└─$ python3 setup.py --help
Common commands: (see '--help-commands' for more)
setup.py build     will build the package underneath 'build/'
setup.py install    will install the package
┌──(kali㉿kali)-[~/twint]
└─$ sudo python3 setup.py install
running install
┌──(kali㉿kali)-[~/twint]
└─$ sudo python3 setup.py build
running build
running build_py
┌──(kali㉿kali)-[~/twint]
└─$ twint -u networkchuck -s "raspberry pi"
Traceback (most recent call last):
File "/usr/local/bin/twint", line 33, in
sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')())
File "/usr/local/bin/twint", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in
from . import run
File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in
from . import datelock, feed, get, output, verbose, storage
File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in
from aiohttp_socks import ProxyConnector, ProxyType
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in
from .connector import (
File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in
from aiohttp.helpers import CeilTimeout  # noqa
ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
Research Issue.
Fix 1: I searched on Google with the error and found,
https://github.com/twintproject/twint/issues/1071
Asked to run the command below to check if there was a .local/bin folder created.
$ echo $PATH | tr -s ":" "\n" | sort
┌──(kali㉿kali)-[~/Downloads/twint]
└─$ echo $PATH | tr -s ":" "\n" | sort 1 ⨯
/bin
/home/kali/.dotnet/tools
/home/kali/.local/bin
/sbin
/usr/bin
/usr/games
/usr/local/bin
/usr/local/games
/usr/local/sbin
/usr/sbin
The advice was to add these two lines to the .zshrc script as I am using zsh shell.
I am having the same issue in bash, so I don't think it matters at all for the moment.
$ export PYTHON_BIN_PATH="$(python3 -m site --user-base)/bin"
$ export PATH="$PATH:$PYTHON_BIN_PATH"
This method didn't work for me.
Still have the error.
Fix2: $ pip3 install yarl --force-reinstall --no-cache-dir
https://stackoverflow.com/questions/64747304/twint-python-library-is-causing-exception-for-search-query?rq=1
A fix was mentioned by a user - "I had the same issue with python3.7 and resolve it by reinstalling yarl, like this way:"
$ pip3 install yarl --force-reinstall --no-cache-dir
This method didn't work for me.
Fix3: To uninstall and reinstall twint with the upgrade.
https://github.com/twintproject/twint/issues/915
    pip3 uninstall twint
    pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
    sudo pip3 install twint

This method didn't work for me.
Fix4: Install on Google Cloud Shell.
Google Cloud New build
    sudo git clone –depth=1 https://github.com/twintproject/twint.git
    cd twint
    sudo pip3 install . -r requirements.txt
    pip3 install twint

This worked for me but once you leave the session, twint seems to be causing the same issue as I had on the Kali machine.
twint -u networkchuck –limit 20 (Did not recognise -limit 20)
twint -u networkchuck -s “raspberry pi”
New Error message.
$ twint -u networkchuck -s crypto -o rightnow.json–json
Traceback (most recent call last):
File "/home/rangersmyth_74/.local/bin/twint", line 10, in
sys.exit(run_as_command())
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/cli.py", line 339, in run_as_command
main()
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/cli.py", line 330, in main
run.Search(c)
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 410, in Search
run(config, callback)
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 329, in run
get_event_loop().run_until_complete(Twint(config).main(callback))
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 36, in init
self.token.refresh()
File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/token.py", line 69, in refresh
raise RefreshTokenException('Could not find the Guest token in HTML')
twint.token.RefreshTokenException: Could not find the Guest token in HTML
Searching for twint.token
https://github.com/twintproject/twint/pull/1146
Fix 5. Add to PATH.
https://issueexplorer.com/issue/twintproject/twint/1189
"" MikeTheScriptKid wrote this answer on 2021-05-08 ""
So I finally got the twint command working in Kali.
I added the following to /etc/environment
/your_user/.local/bin
"" HarryWestFord wrote this answer on 2021-05-10 ""
"sorry for being such a newbie but how excalty do you add this on the command line?
"" hackingbutlegal wrote this answer on 2021-05-15 ""
nano /etc/environment
This didn't work for me.
Found a fix to sort out the token problem
SCRIPT from >> https://github.com/twintproject/twint/pull/1146
import re
import time
import logging as logme
import requests
class TokenExpiryException(Exception):
def init(self, msg):
super().init(msg)
class RefreshTokenException(Exception):
def init(self, msg):
super().init(msg)
class Token:
def init(self, config):
self._session = requests.Session()
self._session.headers.update(
{'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0'})
self.config = config
self._proxies = self._set_proxies()
self._retries = 5
self._timeout = 10
self.url = 'https://twitter.com'
def _set_proxies(self) -> dict:
    settings = [self.config.Proxy_type, self.config.Proxy_host, self.config.Proxy_port]
    if not all(settings):
        logme.debug(f"No proxy in config")
        return {}

    proxy_type = self.config.Proxy_type.lower()
    proxy_val = f"{self.config.Proxy_host}:{self.config.Proxy_port}"
    proxies = {proxy_type: proxy_val}
    if proxy_type == 'http':
        proxies['https'] = proxy_val
    return proxies

def _request(self):
    for attempt in range(self._retries + 1):
        # The request is newly prepared on each retry because of potential cookie updates.
        req = self._session.prepare_request(requests.Request('GET', self.url))
        logme.debug(f'Retrieving {req.url}')
        try:
            if self._proxies:
                r = self._session.send(
                    req,
                    allow_redirects=True,
                    timeout=self._timeout,
                    proxies=self._proxies,
                    verify=False
                )
            else:
                r = self._session.send(req, allow_redirects=True, timeout=self._timeout)
        except requests.exceptions.RequestException as exc:
            if attempt < self._retries:
                retrying = ', retrying'
                level = logme.WARNING
            else:
                retrying = ''
                level = logme.ERROR
            logme.log(level, f'Error retrieving {req.url}: {exc!r}{retrying}')
        else:
            success, msg = (True, None)
            msg = f': {msg}' if msg else ''

            if success:
                logme.debug(f'{req.url} retrieved successfully{msg}')
                return r
        if attempt < self._retries:
            # TODO : might wanna tweak this back-off timer
            sleep_time = 2.0 * 2 ** attempt
            logme.info(f'Waiting {sleep_time:.0f} seconds')
            time.sleep(sleep_time)
    else:
        msg = f'{self._retries + 1} requests to {self.url} failed, giving up.'
        logme.fatal(msg)
        self.config.Guest_token = None
        raise RefreshTokenException(msg)

def refresh(self):
    logme.debug('Retrieving guest token')
    res = self._request()
    match = re.search(r'\("gt=(\d+);', res.text)
    if match:
        logme.debug('Found guest token in HTML')
        self.config.Guest_token = str(match.group(1))
    else:
        self.config.Guest_token = None
        raise RefreshTokenException('Could not find the Guest token in HTML')

I ran this script and got back nothing, no error, but when I did a search it didn't work.
Back to the Drawing Board.
References:
https://github.com/twintproject/twint/wiki/Setup
https://github.com/twintproject/twint/wiki/Configuration
https://github.com/twintproject/twint/issues/468
https://github.com/twintproject/twint/issues/915
https://github.com/twintproject/twint/pull/917
https://github.com/twintproject/twint/pull/944
https://github.com/twintproject/twint/issues/980
https://github.com/twintproject/twint/issues/1114 - Has a fix to upgrade twint && I also repored my issue here.
https://github.com/twintproject/twint/pull/1146 - Token Fix
https://github.com/Altimis/Scweet
http://saka.docsio.net/66892325/scrape-join-dates-user-info-from-a-list-csv-of-twitter-users
http://saka.docsio.net/64747304/twint-python-library-is-causing-exception-for-search-qeruy
https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
https://github.com/himanshudabas/twint/tree/twint-fixes
https://www.kaggle.com/zelinngilo/punya-nadia?scriptVersionId=57423357
https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
https://stackoverflow.com/questions/64747304/twint-python-library-is-causing-exception-for-search-query?rq=1
https://www.bountysource.com/teams/tweep/issues?tracker_ids=65007358
https://issueexplorer.com/issue/twintproject/twint/1266

Solution
The following patch works for me.
# This patch is WTFPL (http://www.wtfpl.net/txt/copying/) and no warranty.
diff --git a/twint/token.py b/twint/token.py
index ae66a24..2eedcee 100644
--- a/twint/token.py
+++ b/twint/token.py
@@ -65,5 +65,30 @@ class Token:
             logme.debug('Found guest token in HTML')
             self.config.Guest_token = str(match.group(1))
         else:
-            self.config.Guest_token = None
-            raise RefreshTokenException('Could not find the Guest token in HTML')
+            headers = {
+                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
+                'authority': 'api.twitter.com',
+                'content-length': '0',
+                'authorization': self.config.Bearer_token,
+                'x-twitter-client-language': 'en',
+                'x-csrf-token': res.cookies.get("ct0"),
+                'x-twitter-active-user': 'yes',
+                'content-type': 'application/x-www-form-urlencoded',
+                'accept': '*/*',
+                'sec-gpc': '1',
+                'origin': 'https://twitter.com',
+                'sec-fetch-site': 'same-site',
+                'sec-fetch-mode': 'cors',
+                'sec-fetch-dest': 'empty',
+                'referer': 'https://twitter.com/',
+                'accept-language': 'en-US',
+            }
+            self._session.headers.update(headers)
+            req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
+            res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
+            match = re.search(r'{"guest_token":"(\d+)"}', res.text)
+            if match:
+                self.config.Guest_token = str(match.group(1))
+            else:
+                self.config.Guest_token = None
+                raise RefreshTokenException('Could not find the Guest token in HTML')

I don't understand session of requests, so the code may be not good.
I hope someone rewrite the patch better and create a pull request.
About the problem
In my environment, this problem has recently begun to occur.
This doesn't happen every time, so if you are lucky, you don't get the error.
The cause is literally literally that twint could not find the Guest token in HTML.
Actually, sometimes token isn't included in HTML recently.
#!/usr/bin/env python3
# This program is WTFPL.
import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the Guest token.
About the solution
In my patch, twint get the Guest token from https://api.twitter.com/1.1/guest/activate.json if could not find the one.
I referred to the code of gallery-dl.

These are the steps I took to install twint and get try and get it to work on 2 versions of Linux and 1 Deb. I have spent 9 hours trying my best, but no fix anywhere, none at all, not one.

Unfortunately, twint hasn't been updated for a long time.
If the problem is new, updating will not solve the problem.
In this case, the problem  is too new to find a solution.
However, the problem may not happen in other environments, so I want reports.

tbh, I've got the same problem like this. My OS is Windows and I'm trying to scrap twitter's tweet in google colab. In the end, RefreshTokenException: Could not find the Guest token in HTML appear everytime I ran twint.run.Search(c). I though that it was just from google colab but after I tried it in Jupyter, the result still the same which is couldn't find the guest token. Any solution for this?

Thank you for the patch @minamotorin which worked for me too! I have created a PR #1322, hopefully any maintainer will be able to merge it soon.

@LinqLover @gabrielruoff
I forgot to update log message, so I updated my patch.
Please confirm it.
The changes are the following two points.

logme.debug('Found guest token in JSON') was added.
Error message of RefreshTokenException was changed.

@minamotorin although i put your solution part in my token.py file, i seem to get this error 'twint.token.RefreshTokenException: Could not find the Guest token in JSON'
can you please guide me through the solution process or even share your token.py file

Additionally to @minamotorin's response, I use this hotfix in my ipynbs. It also adds a response field to the exception, to simplify debugging.
import nest_asyncio
nest_asyncio.apply()
import re
import twint
from twint.token import Token as OriginalToken, RefreshTokenException

class Token(OriginalToken):
  def __init__(self, config):
    super().__init__(config)
    self._session.headers.update({
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'
    })

  def refresh(self):
    res = self._request()
    match = re.search(r'\("gt=(\d+);', res.text)
    if match:
      self.config.Guest_token = str(match.group(1))
    else:
      headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
        'authority': 'api.twitter.com',
        'content-length': '0',
        'authorization': self.config.Bearer_token,
        'x-twitter-client-language': 'en',
        'x-csrf-token': res.cookies.get("ct0"),
        'x-twitter-active-user': 'yes',
        'content-type': 'application/x-www-form-urlencoded',
        'accept': '*/*',
        'sec-gpc': '1',
        'origin': 'https://twitter.com',
        'sec-fetch-site': 'same-site',
        'sec-fetch-mode': 'cors',
        'sec-fetch-dest': 'empty',
        'referer': 'https://twitter.com/',
        'accept-language': 'en-US',
      }
      self._session.headers.update(headers)
      req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
      res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
      match = re.search(r'{"guest_token":"(\d+)"}', res.text)
      if match:
        self.config.Guest_token = str(match.group(1))
      else:
        self.config.Guest_token = None
        exception = RefreshTokenException('Could not find the Guest token in HTML')
        exception.response = res
        raise exception

twint.token.Token = Token

query = "#kaag" #@param {"type": "string"}
since = "2021-01-01" #@param {"type": "string"}
until = "2022-01-08" #@param {"type": "string"}

c = twint.Config()
query = query
c.Search = query
c.Limit = 1000
c.Since, c.Until = since, until
c.Store_object =  True
c.User_full = True
c.Profile_full = True
c.Hide_output = True

try:
  twint.run.Search(c)
except Exception as e:
  response = e.response
  raise

Solution
The following patch works for me.
# This patch is WTFPL (http://www.wtfpl.net/txt/copying/) and no warranty.
diff --git a/twint/token.py b/twint/token.py
index ae66a24..2eedcee 100644
--- a/twint/token.py
+++ b/twint/token.py
@@ -65,5 +65,30 @@ class Token:
             logme.debug('Found guest token in HTML')
             self.config.Guest_token = str(match.group(1))
         else:
-            self.config.Guest_token = None
-            raise RefreshTokenException('Could not find the Guest token in HTML')
+            headers = {
+                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
+                'authority': 'api.twitter.com',
+                'content-length': '0',
+                'authorization': self.config.Bearer_token,
+                'x-twitter-client-language': 'en',
+                'x-csrf-token': res.cookies.get("ct0"),
+                'x-twitter-active-user': 'yes',
+                'content-type': 'application/x-www-form-urlencoded',
+                'accept': '*/*',
+                'sec-gpc': '1',
+                'origin': 'https://twitter.com',
+                'sec-fetch-site': 'same-site',
+                'sec-fetch-mode': 'cors',
+                'sec-fetch-dest': 'empty',
+                'referer': 'https://twitter.com/',
+                'accept-language': 'en-US',
+            }
+            self._session.headers.update(headers)
+            req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
+            res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
+            match = re.search(r'{"guest_token":"(\d+)"}', res.text)
+            if match:
+                logme.debug('Found guest token in JSON')
+                self.config.Guest_token = str(match.group(1))
+            else:
+                self.config.Guest_token = None
+                raise RefreshTokenException('Could not find the Guest token in JSON')

I don't understand session of requests, so the code may be not good. I hope someone rewrite the patch better and create a pull request.
About the problem
In my environment, this problem has recently begun to occur. This doesn't happen every time, so if you are lucky, you don't get the error.
The cause is literally literally that twint could not find the Guest token in HTML. Actually, sometimes token isn't included in HTML recently.
#!/usr/bin/env python3
# This program is WTFPL.
import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>. However, sometimes the result is only })();</script> and missing the Guest token.
About the solution
In my patch, twint get the Guest token from https://api.twitter.com/1.1/guest/activate.json if could not find the one. I referred to the code of gallery-dl.

These are the steps I took to install twint and get try and get it to work on 2 versions of Linux and 1 Deb. I have spent 9 hours trying my best, but no fix anywhere, none at all, not one.

Unfortunately, twint hasn't been updated for a long time. If the problem is new, updating will not solve the problem.
In this case, the problem is too new to find a solution. However, the problem may not happen in other environments, so I want reports.

Happy New Year to you minamotorin and thank you for explaining what was happening.
Happy New Year to everyone also!
I left this problem and worked on another project only coming back now to have another go at installing.
As a complete noob, I understand that the token.py file is the one to edit. I am just not sure what to do next.
Q: Is this the code I have to put into the token.py file?
#!/usr/bin/env python3
This program is WTFPL.
import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])
Q: Or, do I copy and paste the code long code above into the token.py?
I will report back with my findings.
This post will help many I believe and hope.
Thanks in advance.
Ranger.

Additionally to @minamotorin's response, I use this hotfix in my ipynbs. It also adds a response field to the exception, to simplify debugging.
import nest_asyncio
nest_asyncio.apply()
import re
import twint
from twint.token import Token as OriginalToken, RefreshTokenException

class Token(OriginalToken):
  def __init__(self, config):
    super().__init__(config)
    self._session.headers.update({
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'
    })

  def refresh(self):
    res = self._request()
    match = re.search(r'\("gt=(\d+);', res.text)
    if match:
      self.config.Guest_token = str(match.group(1))
    else:
      headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
        'authority': 'api.twitter.com',
        'content-length': '0',
        'authorization': self.config.Bearer_token,
        'x-twitter-client-language': 'en',
        'x-csrf-token': res.cookies.get("ct0"),
        'x-twitter-active-user': 'yes',
        'content-type': 'application/x-www-form-urlencoded',
        'accept': '*/*',
        'sec-gpc': '1',
        'origin': 'https://twitter.com',
        'sec-fetch-site': 'same-site',
        'sec-fetch-mode': 'cors',
        'sec-fetch-dest': 'empty',
        'referer': 'https://twitter.com/',
        'accept-language': 'en-US',
      }
      self._session.headers.update(headers)
      req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
      res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
      match = re.search(r'{"guest_token":"(\d+)"}', res.text)
      if match:
        self.config.Guest_token = str(match.group(1))
      else:
        self.config.Guest_token = None
        exception = RefreshTokenException('Could not find the Guest token in HTML')
        exception.response = res
        raise exception

twint.token.Token = Token

query = "#kaag" #@param {"type": "string"}
since = "2021-01-01" #@param {"type": "string"}
until = "2022-01-08" #@param {"type": "string"}

c = twint.Config()
query = query
c.Search = query
c.Limit = 1000
c.Since, c.Until = since, until
c.Store_object =  True
c.User_full = True
c.Profile_full = True
c.Hide_output = True

try:
  twint.run.Search(c)
except Exception as e:
  response = e.response
  raise

Hey, thanks for this!
What do I need to do so I can do what you suggested please? I am a Noob! lolz.
I have no idea what to do!! lolz, But willing to learn,
I copy and paste all of your code into a file that exists? Then to delete all the code in that file and paste in your code?
Which file and what is the location please?
And am I missing any steps?

I did this to solve the problem
Overwrite your existing token.py file with the content contained here: https://github.com/Museum-Barberini/twint/blob/fix/RefreshTokenException/twint/token.py
See #1075 for a full review of how this worked out for me.

I did this to solve the problem
Overwrite your existing token.py file with the file contents contained here: https://github.com/Museum-Barberini/twint/blob/fix/RefreshTokenException/twint/token.py
See #1075 for a full review of how this worked out for me.

Thanks,its working for me as well

Worked but want to note for me it only retrieves about a week plus a day of tweets.

I did this to solve the problem
Overwrite your existing token.py file with the file contents contained here: https://github.com/Museum-Barberini/twint/blob/fix/RefreshTokenException/twint/token.py
See #1075 for a full review of how this worked out for me.

This works perfectly for me. Thank you

Hey all,
I replaced the code from https://github.com/Museum-Barberini/twint/blob/fix/RefreshTokenException/twint/token.py and replaced it on my file token.py.
I am still getting the error.
I added this code to the top of token.py
#!/usr/bin/env python3
This program is WTFPL.
import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])
I am still getting the error.
I then added this bit of code to the token.py at the bottom.
import nest_asyncio
nest_asyncio.apply()
import re
import twint
from twint.token import Token as OriginalToken, RefreshTokenException
class Token(OriginalToken):
def init(self, config):
super().init(config)
self._session.headers.update({
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36'
})
def refresh(self):
res = self._request()
match = re.search(r'("gt=(\d+);', res.text)
if match:
self.config.Guest_token = str(match.group(1))
else:
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
'authority': 'api.twitter.com',
'content-length': '0',
'authorization': self.config.Bearer_token,
'x-twitter-client-language': 'en',
'x-csrf-token': res.cookies.get("ct0"),
'x-twitter-active-user': 'yes',
'content-type': 'application/x-www-form-urlencoded',
'accept': '/',
'sec-gpc': '1',
'origin': 'https://twitter.com',
'sec-fetch-site': 'same-site',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://twitter.com/',
'accept-language': 'en-US',
}
self._session.headers.update(headers)
req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
match = re.search(r'{"guest_token":"(\d+)"}', res.text)
if match:
self.config.Guest_token = str(match.group(1))
else:
self.config.Guest_token = None
exception = RefreshTokenException('Could not find the Guest token in HTML')
exception.response = res
raise exception
twint.token.Token = Token
query = "#kaag" #@param {"type": "string"}
since = "2021-01-01" #@param {"type": "string"}
until = "2022-01-08" #@param {"type": "string"}
c = twint.Config()
query = query
c.Search = query
c.Limit = 1000
c.Since, c.Until = since, until
c.Store_object =  True
c.User_full = True
c.Profile_full = True
c.Hide_output = True
try:
twint.run.Search(c)
except Exception as e:
response = e.response
raise
Q: Is what I pasted (long code) have to be in a different file?
Can someone point me in the right direction please, as I am very close to fixing this!!

Happy new year too!
@rangersmyth74
If you don't know about syntax of diff, checking this DIFFirence while comparing before patching and after patching helps to understand this patch.

# This patch is WTFPL (http://www.wtfpl.net/txt/copying/) and no warranty.
diff --git a/twint/token.py b/twint/token.py
index ae66a24..2eedcee 100644
--- a/twint/token.py
+++ b/twint/token.py
@@ -65,5 +65,30 @@ class Token:
             logme.debug('Found guest token in HTML')
             self.config.Guest_token = str(match.group(1))
         else:
-            self.config.Guest_token = None
-            raise RefreshTokenException('Could not find the Guest token in HTML')
+            headers = {
+                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
+                'authority': 'api.twitter.com',
+                'content-length': '0',
+                'authorization': self.config.Bearer_token,
+                'x-twitter-client-language': 'en',
+                'x-csrf-token': res.cookies.get("ct0"),
+                'x-twitter-active-user': 'yes',
+                'content-type': 'application/x-www-form-urlencoded',
+                'accept': '*/*',
+                'sec-gpc': '1',
+                'origin': 'https://twitter.com',
+                'sec-fetch-site': 'same-site',
+                'sec-fetch-mode': 'cors',
+                'sec-fetch-dest': 'empty',
+                'referer': 'https://twitter.com/',
+                'accept-language': 'en-US',
+            }
+            self._session.headers.update(headers)
+            req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
+            res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
+            match = re.search(r'{"guest_token":"(\d+)"}', res.text)
+            if match:
+                logme.debug('Found guest token in JSON')
+                self.config.Guest_token = str(match.group(1))
+            else:
+                self.config.Guest_token = None
+                raise RefreshTokenException('Could not find the Guest token in JSON')

The following code doesn't matter for you to solve the problem.

#!/usr/bin/env python3
# This program is WTFPL.
import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

@jonathanmc
Have you tried https://github.com/twintproject/twint/issues/1266#issuecomment-913054208?

I just did and retrieved almost a year's worth before name resolution failure. That might be unrelated - my Internet seemed to go down. I may need to find and increase the timer on twint.
Thank you!

@jonathanmc
You are welcome!

reinstalled it via pip3 install twint

I think you should install twint via pip3 install git+https://github.com/twintproject/twint.git@origin/master#egg=twint (not pip3 install twint) to avoid bugs fixed after latest version of PyPI.
In addition, you can skip step 1 by installing  twint  via pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint (see https://github.com/twintproject/twint/pull/1322#issuecomment-1008441656).

New version of token.py fixed the guest token error, but now I'm back to only being able to get tweets from the last 10 days. I uncommented the line in url.py. This worked before the token.py guest token fix, but now it stopped working.
Anyone else with this problem? Anyone has an idea how to fix this?
I'm on Ubuntu 20.04.3 , Twint 2.1.21, Python 3.8.8, aiohttp-socks 0.7.1

@minamotorin worked for me, thank you so much!

Ran into a profile banner issue key error. #1329 PR raised, please test if it works for you all!

To note,

pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint

uncommented url.py:
('query_source', 'typed_query'),`

Fetches 20 days or so of history on some accounts. Doesn't seem to do the same for all. Same issue as someone above.

Solution
The following patch works for me.
# This patch is WTFPL (http://www.wtfpl.net/txt/copying/) and no warranty.
diff --git a/twint/token.py b/twint/token.py
index ae66a24..2eedcee 100644
--- a/twint/token.py
+++ b/twint/token.py
@@ -65,5 +65,30 @@ class Token:
             logme.debug('Found guest token in HTML')
             self.config.Guest_token = str(match.group(1))
         else:
-            self.config.Guest_token = None
-            raise RefreshTokenException('Could not find the Guest token in HTML')
+            headers = {
+                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
+                'authority': 'api.twitter.com',
+                'content-length': '0',
+                'authorization': self.config.Bearer_token,
+                'x-twitter-client-language': 'en',
+                'x-csrf-token': res.cookies.get("ct0"),
+                'x-twitter-active-user': 'yes',
+                'content-type': 'application/x-www-form-urlencoded',
+                'accept': '*/*',
+                'sec-gpc': '1',
+                'origin': 'https://twitter.com',
+                'sec-fetch-site': 'same-site',
+                'sec-fetch-mode': 'cors',
+                'sec-fetch-dest': 'empty',
+                'referer': 'https://twitter.com/',
+                'accept-language': 'en-US',
+            }
+            self._session.headers.update(headers)
+            req = self._session.prepare_request(requests.Request('POST', 'https://api.twitter.com/1.1/guest/activate.json'))
+            res = self._session.send(req, allow_redirects=True, timeout=self._timeout)
+            match = re.search(r'{"guest_token":"(\d+)"}', res.text)
+            if match:
+                logme.debug('Found guest token in JSON')
+                self.config.Guest_token = str(match.group(1))
+            else:
+                self.config.Guest_token = None
+                raise RefreshTokenException('Could not find the Guest token in JSON')

I don't understand session of requests, so the code may be not good. I hope someone rewrite the patch better and create a pull request.
About the problem
In my environment, this problem has recently begun to occur. This doesn't happen every time, so if you are lucky, you don't get the error.
The cause is literally literally that twint could not find the Guest token in HTML. Actually, sometimes token isn't included in HTML recently.
#!/usr/bin/env python3
# This program is WTFPL.
import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>. However, sometimes the result is only })();</script> and missing the Guest token.
About the solution
In my patch, twint get the Guest token from https://api.twitter.com/1.1/guest/activate.json if could not find the one. I referred to the code of gallery-dl.

These are the steps I took to install twint and get try and get it to work on 2 versions of Linux and 1 Deb. I have spent 9 hours trying my best, but no fix anywhere, none at all, not one.

Unfortunately, twint hasn't been updated for a long time. If the problem is new, updating will not solve the problem.
In this case, the problem is too new to find a solution. However, the problem may not happen in other environments, so I want reports.

I tried this but I still get the error :/

@minamotorin and everyone in this thread, it seems that these hotfixes are not working well if I am scraping a large amount of users simultaneously. Earlier tonight, twint.run.Profile was working normally, but after I began to multiprocess scrape tweets out of 8000 accounts, twint.token.RefreshTokenException: Could not find the Guest token in HTML popped up, and all future attempts will always result in this.
I am using the twint-2.1.21 version from @minamotorin's own repo. Does this mean I am subject to some anti-scraping mechanism of twitter?

I really love this app and I have only used it once. But now I want to use it and I can't. I have searched so much that my eyes are sore, why am I getting these errors and why can't I find a fix for it!! hehe...
Linux Kali-ROG 5.14.0-kali4-amd64 #1 SMP Debian 5.14.16-1kali1 (2021-11-05) x86_64 GNU/Linux
Please help not just me but anyone who searches the 400 pages of information to find an answer. 9 hours of work to try this, the time I will never get back and I just want to chuck in Linux altogether and do anything else.

[] Python version is 3.6;
[] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;

These are the steps I took to install twint and get try and get it to work on 2 versions of Linux and 1 Deb. I have spent 9 hours trying my best, but no fix anywhere, none at all, not one.
What errors am I getting, all listed below
1: Install on Kali. This did not work for me.
Linux Kali-ROG 5.14.0-kali4-amd64 #1 SMP Debian 5.14.16-1kali1 (2021-11-05) x86_64 GNU/Linux
Instructions for install : https://github.com/twintproject/twint
    git clone –depth=1 https://github.com/twintproject/twint.git
    cd twint
    pip3 install . -r requirements.txt

Try and run these commands to see if it works.
   $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

2: Install on Kali AWS. This did not work for me.
Linux kali 5.15.0-kali2-cloud-amd64 #1 SMP Debian 5.15.5-2kali2 (2021-12-22) x86_64 GNU/Linux
It seems that twint and AWS do not work due to
Instructions for install : https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
    sudo git clone –depth=1 https://github.com/twintproject/twint.git
    sudo cd twint
    sudo pip3 install . -r requirements.txt
    sudo pip3 install twint

Try and run these commands to see if it works.
    $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

3: Install on Google Cloud Shell and This worked for me.
Lightbox Linux cs-899333161534-default-boost-h8jtm 5.10.68+ #1 SMP Wed Dec 1 10:07:21 UTC 2021 x86_64 GNU/Linux
Instructions for install: https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
FYI 1 - Google didn't like the command git clone --depth=1
So I removed it and it installed.
FY 2 - Using pip3 install twint command said all was installed, but I added sudo
$ sudo git clone https://github.com/twintproject/twint.git $ ls $ cd twint $ ls $ sudo pip3 install . -r requirements.txt $ sudo pip3 install twint
Ran commands to see if it works.
    $ twint -s pineapple
    $ twint -u networkchuck -s "raspberry pi"

I was able to run twint until I logged out and logged back in, it didn't work anymore.
The Kali twint install journey.
My Install on Kali Linux.
┌──(kali㉿kali)-[~] └─$ git clone --depth=1 https://github.com/twintproject/twint.git Cloning into 'twint'... remote: Enumerating objects: 47, done. remote: Counting objects: 100% (47/47), done. remote: Compressing objects: 100% (44/44), done. remote: Total 47 (delta 3), reused 14 (delta 0), pack-reused 0 Receiving objects: 100% (47/47), 42.95 KiB | 1.59 MiB/s, done. Resolving deltas: 100% (3/3), done.
┌──(kali㉿kali)-[~] └─$ ls Desktop Documents Downloads Music Pictures Public Templates Videos bash.txt nmap twint
┌──(kali㉿kali)-[~] └─$ cd twint
┌──(kali㉿kali)-[~/twint] └─$cd twint
┌──(kali㉿kali)-[~/twint] └─$ pip3 install . -r requirements.txt Command 'pip3' not found, but can be installed with: sudo apt install python3-pip Do you want to install it? (N/y)y sudo apt install python3-pip
┌──(kali㉿kali)-[~/twint] └─$ pip3 install . -r requirements.txt
WARNING: The script twint is installed in '/home/kali/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed aiohttp-socks-0.4.1 cchardet-2.1.7 dataclasses-0.6 elasticsearch-7.16.2 fake-useragent-0.1.11 geographiclib-1.52 geopy-2.2.0 googletransx-2.4.2 pandas-1.3.5 schedule-1.1.0 twint-2.1.21
(I added two commands for the path later on in the install, see far below).
┌──(kali㉿kali)-[~/.local] └─$ cd bin
┌──(kali㉿kali)-[~/.local/bin] └─$ ls cchardetect translate twint
┌──(kali㉿kali)-[~/.local/bin] └─$ twint -u networkchuck Traceback (most recent call last): File "/usr/local/bin/twint", line 33, in sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')()) File "/usr/local/bin/twint", line 25, in importlib_load_entry_point return next(matches).load() File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 972, in _find_and_load_unlocked File "", line 228, in _call_with_frames_removed File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "", line 850, in exec_module File "", line 228, in _call_with_frames_removed File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in from . import run File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in from . import datelock, feed, get, output, verbose, storage File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in from aiohttp_socks import ProxyConnector, ProxyType File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in from .connector import ( File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in from aiohttp.helpers import CeilTimeout # noqa ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
┌──(kali㉿kali)-[~/.local/bin] └─$ echo $SHELL
/bin/zsh
┌──(kali㉿kali)-[~/.local/bin] └─$ bash
┌──(kali㉿kali)-[~/.local/bin] └─$ myip Command 'myip' not found, did you mean: command 'mzip' from deb mtools Try: sudo apt install
┌──(kali㉿kali)-[~/.local/bin] └─$ twint -u networkchuck -s "raspberry pi" Traceback (most recent call last): File "/usr/local/bin/twint", line 33, in sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')()) File "/usr/local/bin/twint", line 25, in importlib_load_entry_point return next(matches).load() File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 972, in _find_and_load_unlocked File "", line 228, in _call_with_frames_removed File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "", line 850, in exec_module File "", line 228, in _call_with_frames_removed File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in from . import run File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in from . import datelock, feed, get, output, verbose, storage File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in from aiohttp_socks import ProxyConnector, ProxyType File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in from .connector import ( File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in from aiohttp.helpers import CeilTimeout # noqa ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
──(kali㉿kali)-[~/.local/bin] └─$ cd ..
┌──(kali㉿kali)-[~/.local] └─$ cd.. cd..: command not found
┌──(kali㉿kali)-[~/.local] └─$ cd ..
┌──(kali㉿kali)-[~] └─$
┌──(kali㉿kali)-[~] └─$ twint -u networkchuck -s "raspberry pi" Command 'twint' not found, did you mean: command 'twine' from deb twine Try: sudo apt install
┌──(kali㉿kali)-[~/twint] └─$ python3 Python 3.9.9 (main, Dec 16 2021, 23:13:29) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.

┌──(kali㉿kali)-[~/twint] └─$ twint -u networkchuck -s "raspberry pi" Command 'twint' not found, did you mean: command 'twine' from deb twine Try: sudo apt install
┌──(kali㉿kali)-[~] └─$ sudo apt update && sudo apt upgrade
┌──(kali㉿kali)-[~] └─$ systemctl reboot -i
┌──(kali㉿kali)-[~/twint] └─$cd twint
┌──(kali㉿kali)-[~/twint] └─$ chmod +x setup.py
┌──(kali㉿kali)-[~/twint] └─$ python3 setup.py usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: no commands supplied ──(kali㉿kali)-[~/twint] └─$ python3 setup.py --help Common commands: (see '--help-commands' for more)
setup.py build will build the package underneath 'build/' setup.py install will install the package
┌──(kali㉿kali)-[~/twint] └─$ sudo python3 setup.py install
running install
┌──(kali㉿kali)-[~/twint] └─$ sudo python3 setup.py build running build running build_py
┌──(kali㉿kali)-[~/twint] └─$ twint -u networkchuck -s "raspberry pi" Traceback (most recent call last): File "/usr/local/bin/twint", line 33, in sys.exit(load_entry_point('twint==2.1.21', 'console_scripts', 'twint')()) File "/usr/local/bin/twint", line 25, in importlib_load_entry_point return next(matches).load() File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 972, in _find_and_load_unlocked File "", line 228, in _call_with_frames_removed File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "", line 850, in exec_module File "", line 228, in _call_with_frames_removed File "/home/kali/.local/lib/python3.9/site-packages/twint/init.py", line 14, in from . import run File "/home/kali/.local/lib/python3.9/site-packages/twint/run.py", line 4, in from . import datelock, feed, get, output, verbose, storage File "/home/kali/.local/lib/python3.9/site-packages/twint/get.py", line 12, in from aiohttp_socks import ProxyConnector, ProxyType File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/init.py", line 5, in from .connector import ( File "/home/kali/.local/lib/python3.9/site-packages/aiohttp_socks/connector.py", line 8, in from aiohttp.helpers import CeilTimeout # noqa ImportError: cannot import name 'CeilTimeout' from 'aiohttp.helpers' (/usr/lib/python3/dist-packages/aiohttp/helpers.py)
Research Issue.
Fix 1: I searched on Google with the error and found,
#1071
Asked to run the command below to check if there was a .local/bin folder created.
$ echo $PATH | tr -s ":" "\n" | sort
┌──(kali㉿kali)-[~/Downloads/twint] └─$ echo $PATH | tr -s ":" "\n" | sort 1 ⨯ /bin /home/kali/.dotnet/tools /home/kali/.local/bin /sbin /usr/bin /usr/games /usr/local/bin /usr/local/games /usr/local/sbin /usr/sbin
The advice was to add these two lines to the .zshrc script as I am using zsh shell. I am having the same issue in bash, so I don't think it matters at all for the moment.
$ export PYTHON_BIN_PATH="$(python3 -m site --user-base)/bin"
$ export PATH="$PATH:$PYTHON_BIN_PATH"
This method didn't work for me. Still have the error.
Fix2: $ pip3 install yarl --force-reinstall --no-cache-dir
https://stackoverflow.com/questions/64747304/twint-python-library-is-causing-exception-for-search-query?rq=1
A fix was mentioned by a user - "I had the same issue with python3.7 and resolve it by reinstalling yarl, like this way:"
$ pip3 install yarl --force-reinstall --no-cache-dir
This method didn't work for me.
Fix3: To uninstall and reinstall twint with the upgrade.
#915
    pip3 uninstall twint
    pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
    sudo pip3 install twint

This method didn't work for me.
Fix4: Install on Google Cloud Shell.
Google Cloud New build
    sudo git clone –depth=1 https://github.com/twintproject/twint.git
    cd twint
    sudo pip3 install . -r requirements.txt
    pip3 install twint

This worked for me but once you leave the session, twint seems to be causing the same issue as I had on the Kali machine.
twint -u networkchuck –limit 20 (Did not recognise -limit 20)
twint -u networkchuck -s “raspberry pi”
New Error message.
$ twint -u networkchuck -s crypto -o rightnow.json–json
Traceback (most recent call last): File "/home/rangersmyth_74/.local/bin/twint", line 10, in sys.exit(run_as_command()) File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/cli.py", line 339, in run_as_command main() File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/cli.py", line 330, in main run.Search(c) File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 410, in Search run(config, callback) File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 329, in run get_event_loop().run_until_complete(Twint(config).main(callback)) File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/run.py", line 36, in init self.token.refresh() File "/home/rangersmyth_74/.local/lib/python3.7/site-packages/twint/token.py", line 69, in refresh raise RefreshTokenException('Could not find the Guest token in HTML') twint.token.RefreshTokenException: Could not find the Guest token in HTML
Searching for twint.token
#1146 Fix 5. Add to PATH. https://issueexplorer.com/issue/twintproject/twint/1189
"" MikeTheScriptKid wrote this answer on 2021-05-08 "" So I finally got the twint command working in Kali. I added the following to /etc/environment /your_user/.local/bin
"" HarryWestFord wrote this answer on 2021-05-10 "" "sorry for being such a newbie but how excalty do you add this on the command line?
"" hackingbutlegal wrote this answer on 2021-05-15 ""
nano /etc/environment
This didn't work for me.
Found a fix to sort out the token problem
SCRIPT from >> #1146
import re import time import logging as logme import requests
class TokenExpiryException(Exception): def init(self, msg): super().init(msg)
class RefreshTokenException(Exception): def init(self, msg): super().init(msg)
class Token: def init(self, config): self._session = requests.Session() self._session.headers.update( {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0'}) self.config = config self._proxies = self._set_proxies() self._retries = 5 self._timeout = 10 self.url = 'https://twitter.com'
def _set_proxies(self) -> dict:
    settings = [self.config.Proxy_type, self.config.Proxy_host, self.config.Proxy_port]
    if not all(settings):
        logme.debug(f"No proxy in config")
        return {}

    proxy_type = self.config.Proxy_type.lower()
    proxy_val = f"{self.config.Proxy_host}:{self.config.Proxy_port}"
    proxies = {proxy_type: proxy_val}
    if proxy_type == 'http':
        proxies['https'] = proxy_val
    return proxies

def _request(self):
    for attempt in range(self._retries + 1):
        # The request is newly prepared on each retry because of potential cookie updates.
        req = self._session.prepare_request(requests.Request('GET', self.url))
        logme.debug(f'Retrieving {req.url}')
        try:
            if self._proxies:
                r = self._session.send(
                    req,
                    allow_redirects=True,
                    timeout=self._timeout,
                    proxies=self._proxies,
                    verify=False
                )
            else:
                r = self._session.send(req, allow_redirects=True, timeout=self._timeout)
        except requests.exceptions.RequestException as exc:
            if attempt < self._retries:
                retrying = ', retrying'
                level = logme.WARNING
            else:
                retrying = ''
                level = logme.ERROR
            logme.log(level, f'Error retrieving {req.url}: {exc!r}{retrying}')
        else:
            success, msg = (True, None)
            msg = f': {msg}' if msg else ''

            if success:
                logme.debug(f'{req.url} retrieved successfully{msg}')
                return r
        if attempt < self._retries:
            # TODO : might wanna tweak this back-off timer
            sleep_time = 2.0 * 2 ** attempt
            logme.info(f'Waiting {sleep_time:.0f} seconds')
            time.sleep(sleep_time)
    else:
        msg = f'{self._retries + 1} requests to {self.url} failed, giving up.'
        logme.fatal(msg)
        self.config.Guest_token = None
        raise RefreshTokenException(msg)

def refresh(self):
    logme.debug('Retrieving guest token')
    res = self._request()
    match = re.search(r'\("gt=(\d+);', res.text)
    if match:
        logme.debug('Found guest token in HTML')
        self.config.Guest_token = str(match.group(1))
    else:
        self.config.Guest_token = None
        raise RefreshTokenException('Could not find the Guest token in HTML')

I ran this script and got back nothing, no error, but when I did a search it didn't work.
Back to the Drawing Board.
References:
https://github.com/twintproject/twint/wiki/Setup
https://github.com/twintproject/twint/wiki/Configuration
#468
#915
#917
#944
#980
#1114 - Has a fix to upgrade twint && I also repored my issue here.
#1146 - Token Fix
https://github.com/Altimis/Scweet
http://saka.docsio.net/66892325/scrape-join-dates-user-info-from-a-list-csv-of-twitter-users
http://saka.docsio.net/64747304/twint-python-library-is-causing-exception-for-search-qeruy
https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
https://github.com/himanshudabas/twint/tree/twint-fixes
https://www.kaggle.com/zelinngilo/punya-nadia?scriptVersionId=57423357
https://www.geeksforgeeks.org/how-to-use-twint-osint-tool-on-google-cloud-console/
https://stackoverflow.com/questions/64747304/twint-python-library-is-causing-exception-for-search-query?rq=1
https://www.bountysource.com/teams/tweep/issues?tracker_ids=65007358
https://issueexplorer.com/issue/twintproject/twint/1266

⚙Install & Usage:
$ sudo apt update
$ git clone --depth=1 https://github.com/twintproject/twint.git
$ ls
$ cd twint
$ ls
$ pip3 install . -r requirements.txt
$ sudo python3 setup.py install
$ sudo twint -h
$ sudo -i
$ twint -u Andela
The video link is given here- https://youtu.be/Jk5TF-yEZZ4


⚠ Avatar API Endpoint has degraded performance

In ddd3452, Avatar API Endpoint (https://avatar.roblox.com/v1/avatar-rules) experienced degraded performance:

HTTP code: 200
Response time: 3662 ms

Resolved: Avatar API Endpoint performance has improved in db7ab41 after 11 minutes.


Don’t pretend 32 bit is the norm today

64 bit hardware and OSes aren’t that new anymore.

merge?


Update to-request-vpc-connectivity.adoc

Updating this based on customer feedback - current format was causing confusion. New format addresses those concerns.
This also supersedes the change mentioned in #284

The headers should be just IPsec Connections, VPC Peering Connections, or Direct Connect Connections. The new or update part makes it too wordy, and that is called out in the points below.

Because these are tasks, we need to include the imperative. I tried to make them a bit shorter.


Update model:core to 1.4.13

Updates com.softwaremill.sttp.model:core from 1.4.11 to 1.4.13.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.softwaremill.sttp.model", artifactId = "core" } ]

labels: library-update, semver-patch

Superseded by #414.


Advanced seasons not working with essentials/cmi

Describe the bug
It shows it is installed but nothing is working with Advanced Seasons.  I was told by Moderator Vision to report the issue here so the devs can solve it for me. I tried typing /as addworld world and restarted the server and it still doesn't look like it's working.
How to reproduce
Not sure, installed Advanced Seasons along with dependency, it looks like everything is installed properly but plugin doesn't appear to be working or showing up visually.
Screenshots / Videos

Server Log
No response

Can someone please help with this issue I am experiencing?

Hii @cmongelli provide your startup log.

Hi @SirJben sorry for the delay. Where exactly can I find the startup log to share with you?

Hii @cmongelli provide your startup log.

Could you please tell me where I can find that so I can share it?

You can paste the file here with https://mcpaste.io/ or open a Discord Ticket and tag me there.

Where do I find that file to paste?
On Mon, Jun 24, 2024 at 12:14 AM SirJben @.***> wrote:

You can paste the file here with https://mcpaste.io/ or open a Discord
Ticket and tag me there.
—
Reply to this email directly, view it on GitHub
https://github.com/AdvancedPlugins/Seasons/issues/116#issuecomment-2185569735,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANMTIFFINX647O3H5SXJ25DZI6MLJAVCNFSM6AAAAABJM5FAG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGU3DSNZTGU
.
You are receiving this because you were mentioned.Message ID:
@.***>

@Ynverxe @SirJben Not sure if I did it correctly or with the correct latest log but I pasted it to the mcpaste.io. This is what I found off our server in the logs folder. Please let me know if there is anything else I need to do. Thank you for the help!

You need to past the log in mcpaste.io, save the past and copy the link of your browser and paste the link here.


[#1434] fix(core): Fix the bug of rename entity

What changes were proposed in this pull request?
Use the overwrite strategy when changing the content of name-id mappings in case of possible bugs.
Why are the changes needed?
If the name already existed in the name-id mappings, it would encounter problems.
Fix: #1434
Does this PR introduce any user-facing change?
N/A.
How was this patch tested?
Add UT and IT.

Can you explain more about the problem here?

Can you explain more about the problem here?

The core reason is that deleting an entity does not remove the name-id mapping of it.  Assuming we create two metalakes named a and b, then there exists the following name-id mappings:
a --> 1
b --> 2
When we remove metalake a, b and the mapping relationship will not be dropped. When we put a new metalake named
a, it's OK.
Finally, when we attempt to rename a to b, we should update the mapping relationship and add a new relationship.
b -> 1, however as key b exists in the name-id mapping, we need to use the overwrite policy to update it.


main.py (p4c-bmv2) stuck at "add padding"

when compile p4 source code which is similar to TLV_parsing , p4c-bm stuck after output
parsing successful
semantic checking successful
Header type standard_metadata_t not byte-aligned , adding padding

Thanks for the help ,
John

I don't understand this issue. Why did you rename it to "sorry about the mistake"? That's a pretty useless title.
If you think the compiler is getting stuck on a particular P4 input, please make sure that you have the latest version of p4-hlir and p4c-bm. If the problem persists, please send your P4 source so we can reproduce the issue.

the title is about my network :
After posted this issue I refresh the page , but found two same post on the list ( actually there is only one post , maybe it because my network latency). So I closed one post and changed it's title to "sorry about the mistake" (╯﹏╰)b . Besides English is not my mother language ,if some phrase of the post is aggressive , please dismiss it which is absolutely not my real meaning.
After upgrade p4-hli and p4c-bm the problem is still exist. Here is my p4 source code , and the command I used to compile is sudo p4c-bmv2 --json simple_router.json simple_router_p4src/simple_router.p4  .
Thanks guys. John.

I was able to run p4c-bmv2 on your code without any issue. Are you really certain that you are using the latest p4c-bm & p4-hlir. Because I remember addressing this kind of issues in the past: https://github.com/p4lang/p4c-bm/pull/80

Yes , I'm sure . I just recompiled and reinstalled p4c-bm & p4-hlir again . Maybe because your p4c-bmv2 test just on a changing branch . I created a new branch which still have the problem and I'll not push it before your p4c-bmv2 test on it .

I wondering why the TLV_paring  could work while my code couldn't . So I add tcp options parser from 1 to 8 (the final code looks like) and found that as the tcp options's count  increasing the time p4c-bmv2  used to parsing increasing rapidly , seems like a exponential relationship . The follow is the result

tcp options count
time

1 ~ 4
very quick

5
1 s

6
10 s

7
5 M

8
...

I was able to reproduce this by using the branch reference you provided. This was an issue in p4-hlir, which I fixed with https://github.com/p4lang/p4-hlir/pull/30. Thanks for reporting this issue.


DOC: Remove duplicated Indexing Changes section

"Indexing Changes" section is duplicated in what's new.

http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html

already fixed in #9644


Fix subject area to use the new level attribute type in Confidentiality classification

Fix subject area to use the new level attribute type in Confidentiality classification. level has been changed from an enum to an integer. The subject area code is now project so needs updating

Subject Area OMAS should augment the confidentiality level with values from the confidentiality level entity - so it is more descriptive.
It should  police the integer values that are input to ensure they match ConfidentialityLevel entity content.
It should tolerate on read values that do not have a matching Confidentiality level entity


unwinder/native: Increase size of mappings

This commit increases the maximum mappings a process without FP can have. This limitation affected a small number of users, but it was still a blocker for some.
Additionally, rather than a linear search, we switch to using binary search on the mappings as otherwise it's too many instructions for the BPF verifier to check.
Test Plan
=== RUN   TestCPUProfilerWorks/mixed_mode_unwinding
=== RUN   TestCPUProfilerWorks/unwinder_metrics_work
--- PASS: TestCPUProfilerWorks (20.79s)
    --- PASS: TestCPUProfilerWorks/dwarf_unwinding (0.08s)
    --- PASS: TestCPUProfilerWorks/fp_unwinding (0.67s)
    --- PASS: TestCPUProfilerWorks/mixed_mode_unwinding (0.04s)
    --- PASS: TestCPUProfilerWorks/unwinder_metrics_work (0.00s)
PASS
ok      github.com/parca-dev/parca-agent/test/integration       20.810s

and kernel tests pass

I thought of generalising binary search for both mappings and unwind information but unfortunately, the result would be harder to maintain (macros in C, ugh) than just having two implementations. The code is compact enough that making modifications to both, if needed, shouldn't be a big overhead. We should be careful not to forget to keep both binary searches updated, though.

Maybe we should comment on the two implementations that they need to be kept in sync? Otherwise lgtm.

They don't necessarily have to be kept 100% in sync, e.g. if we want to change something for one of them in the future that only affects mappings or unwind info, it's mostly in case we find out that the current algorithm has an off by one or integer overflow or something similar.
Now that I am writing this, we might have one, will prep a fix!

Can add the comment later if you feel strongly about it!

Nope all good, thanks for the explanation!


Bluetooth command failed (code: 1, error: Invalid handle)  on ATC

I have 2 sensors, one is original and one is flashed with the Telink flasher.
The flashed sensor gives: Bluetooth command failed (code: 1, error: Invalid handle)  on ATC
I can't find out why this happens
Can you help me out?
$sudo hcitool lescan
A4:C1:38:F5:53:CD LYWSD03MMC
A4:C1:38:C8:E0:25 ATC_C8E025
loginfo:
2021-07-31 09:05:11 INFO     TH_IDX:121
2021-07-31 09:05:14 ERROR    Bluetooth command failed (code: 1, error: Invalid handle)
2021-07-31 09:05:14 INFO     TH_IDX:122
2021-07-31 09:05:22 INFO     Temp: 22.84
2021-07-31 09:05:22 INFO     Hum: 68
2021-07-31 09:05:22 INFO     Voltage: 2.973
2021-07-31 09:05:22 INFO     Battery level: 87
2021-07-31 09:05:22 INFO     Comfort type: 1
2021-07-31 09:05:22 INFO     The request is http://XXX.XXX.X.XXX:XXXX/json.htm?Type=command&param=udevice&idx=122&nvalue=0&svalue=22.84;68;1&battery=87
2021-07-31 09:05:22 INFO     The response is <Response [200]>
2021-07-31 09:05:22 INFO     The request is http://XXX.XXX.X.XXX:XXXX/json.htm?type=command&param=udevice&idx=123&nvalue=0&svalue=2.973
2021-07-31 09:05:22 INFO     The response is <Response [200]>

I have the same problem after I flashed my devices with the ATC frimware.
Running 4 sensors, this is my log:
2021-08-30 23:00:02 INFO     Temp: 23.8
2021-08-30 23:00:02 INFO     Hum: 53
2021-08-30 23:00:02 INFO     Comfort type: 1
2021-08-30 23:00:02 INFO     Battery level: 66
2021-08-30 23:00:02 INFO     Voltage: 2.81
2021-08-30 23:00:03 INFO     Temp: 22.7
2021-08-30 23:00:03 INFO     Hum: 57
2021-08-30 23:00:03 INFO     Comfort type: 1
2021-08-30 23:00:03 INFO     Battery level: 40
2021-08-30 23:00:03 INFO     Voltage: 2.584
2021-08-30 23:00:03 INFO     Temp: 21.4
2021-08-30 23:00:03 INFO     Hum: 60
2021-08-30 23:00:03 INFO     Comfort type: 1
2021-08-30 23:00:03 INFO     Battery level: 65
2021-08-30 23:00:03 INFO     Voltage: 2.797
2021-08-30 23:00:03 INFO     Temp: 23.5
2021-08-30 23:00:03 INFO     Hum: 55
2021-08-30 23:00:03 INFO     Comfort type: 1
2021-08-30 23:00:03 INFO     Battery level: 59
2021-08-30 23:00:03 INFO     Voltage: 2.747
2021-08-30 23:00:04 INFO     Temp: 23.8
2021-08-30 23:00:04 INFO     Hum: 53
2021-08-30 23:00:04 INFO     Comfort type: 1
2021-08-30 23:00:04 INFO     Battery level: 66
2021-08-30 23:00:04 INFO     Voltage: 2.81
2021-08-30 23:00:05 INFO     Temp: 22.7
2021-08-30 23:00:05 INFO     Hum: 57
2021-08-30 23:00:05 INFO     Comfort type: 1
2021-08-30 23:00:05 INFO     Battery level: 40
2021-08-30 23:00:05 INFO     Voltage: 2.584
2021-08-30 23:00:05 INFO     Temp: 21.4
2021-08-30 23:00:05 INFO     Hum: 60
2021-08-30 23:00:05 INFO     Comfort type: 1
2021-08-30 23:00:05 INFO     Battery level: 65
2021-08-30 23:00:05 INFO     Voltage: 2.797
2021-08-30 23:00:05 INFO     Temp: 23.5
2021-08-30 23:00:05 INFO     Hum: 55
2021-08-30 23:00:05 INFO     Comfort type: 1
2021-08-30 23:00:05 INFO     Battery level: 59
2021-08-30 23:00:05 INFO     Voltage: 2.747
2021-08-30 23:00:06 INFO     Temp: 23.8
2021-08-30 23:00:06 INFO     Hum: 53
2021-08-30 23:00:06 INFO     Comfort type: 1
2021-08-30 23:00:06 INFO     Battery level: 66
2021-08-30 23:00:06 INFO     Voltage: 2.81
2021-08-30 23:00:07 INFO     Temp: 22.7
2021-08-30 23:00:07 INFO     Hum: 57
2021-08-30 23:00:07 INFO     Comfort type: 1
2021-08-30 23:00:07 INFO     Battery level: 40
2021-08-30 23:00:07 INFO     Voltage: 2.584
2021-08-30 23:00:07 INFO     Temp: 21.4
2021-08-30 23:00:07 INFO     Hum: 60
2021-08-30 23:00:07 INFO     Comfort type: 1
2021-08-30 23:00:07 INFO     Battery level: 65
2021-08-30 23:00:07 INFO     Voltage: 2.797
2021-08-30 23:00:08 INFO     Temp: 23.8
2021-08-30 23:00:08 INFO     Hum: 53
2021-08-30 23:00:08 INFO     Comfort type: 1
2021-08-30 23:00:08 INFO     Battery level: 66
2021-08-30 23:00:08 INFO     Voltage: 2.81
2021-08-30 23:00:09 INFO     Temp: 22.7
2021-08-30 23:00:09 INFO     Hum: 57
2021-08-30 23:00:09 INFO     Comfort type: 1
2021-08-30 23:00:09 INFO     Battery level: 40
2021-08-30 23:00:09 INFO     Voltage: 2.584
2021-08-30 23:00:09 INFO     Temp: 21.4
2021-08-30 23:00:09 INFO     Hum: 60
2021-08-30 23:00:09 INFO     Comfort type: 1
2021-08-30 23:00:09 INFO     Battery level: 65
2021-08-30 23:00:09 INFO     Voltage: 2.797
2021-08-30 23:00:09 INFO     Temp: 23.5
2021-08-30 23:00:09 INFO     Hum: 55
2021-08-30 23:00:09 INFO     Comfort type: 1
2021-08-30 23:00:09 INFO     Battery level: 59
2021-08-30 23:00:09 INFO     Voltage: 2.747
2021-08-30 23:00:10 INFO     Temp: 23.8
2021-08-30 23:00:10 INFO     Hum: 53
2021-08-30 23:00:10 INFO     Comfort type: 1
2021-08-30 23:00:10 INFO     Battery level: 66
2021-08-30 23:00:10 INFO     Voltage: 2.81
2021-08-30 23:00:11 INFO     Temp: 21.4
2021-08-30 23:00:11 INFO     Hum: 60
2021-08-30 23:00:11 INFO     Comfort type: 1
2021-08-30 23:00:11 INFO     Battery level: 65
2021-08-30 23:00:11 INFO     Voltage: 2.797
2021-08-30 23:00:11 INFO     Temp: 22.7
2021-08-30 23:00:11 INFO     Hum: 57
2021-08-30 23:00:11 INFO     Comfort type: 1
2021-08-30 23:00:11 INFO     Battery level: 40
2021-08-30 23:00:11 INFO     Voltage: 2.584
2021-08-30 23:00:11 INFO     Temp: 23.5
2021-08-30 23:00:11 INFO     Hum: 55
2021-08-30 23:00:11 INFO     Comfort type: 1
2021-08-30 23:00:11 INFO     Battery level: 59
2021-08-30 23:00:11 INFO     Voltage: 2.747
2021-08-30 23:00:11 INFO     TH_IDX:27514
2021-08-30 23:00:14 ERROR    Bluetooth command failed (code: 1, error: Invalid handle)
2021-08-30 23:00:14 INFO     TH_IDX:27515
2021-08-30 23:00:18 ERROR    Bluetooth command failed (code: 1, error: Invalid handle)
2021-08-30 23:00:18 INFO     TH_IDX:27516
2021-08-30 23:00:24 ERROR    Bluetooth command failed (code: 1, error: Invalid handle)
2021-08-30 23:00:24 INFO     TH_IDX:27517
2021-08-30 23:00:30 ERROR    Bluetooth command failed (code: 1, error: Invalid handle)

I switched to https://github.com/JsBergbau/MiTemperature2 this works with the ATC frimware !

Hello, i've corrected this issue, you can find it here:
https://github.com/Jaj0/xiaomi-mi-lywsd03mmc


Spike to findout where is stored a user preference given its profile on WebADE

Find out where (tables and columns) and how user's profile are stored inside WebADE.

@webgismd is there a WEBADE SME that we can reach out to for guidance?

More info

WebADE repo: https://apps.nrs.gov.bc.ca/int/stash/projects/INFRA/repos/webade-vm/browse?

Basically, all WebADE and ADAM SQL scripts can be found here: https://apps.nrs.gov.bc.ca/int/stash/projects/INFRA/repos/webade-vm/browse/flyway-webade/sql

I am curious what user preferences you are looking for.. there are 17 roles in Webade for SPAR. Are you looking for that list or a list of users in each role? I wasn't sure if this is what is being asked?

@webgismd thanks for the question. We want to know:

Roles, profiles and orgs of SPAR test users
Default agency information (client number, acronym, location code) for the profiles and orgs

We now have access to files containing profile users count and organizations information. Let's call this done.


WIP: Add a validating webhook to EdgeDevice

Prevent irrelevant or conflicting device specifications.
Signed-off-by: Roy Golan<EMAIL_ADDRESS>ECOPROJECT-347

@rgolangh - how is the work going on that feature? It seems to be needed for #148.

@rgolangh will you be able to finalize this PR?


tls/x509/certpool: introduce CertPool.Reset()

to remove all certificates from the store

Summary by CodeRabbit

New Features

Introduced a new Reset method for the CertPool struct, enhancing its accessibility and thread safety.

Bug Fixes

Preserved original functionality with the introduction of unsafeReset, ensuring reliable internal state modifications.

rebased to resolve conflict


manage orphan serials to map a config to a new board often fails to map 7segment displays

When you swap an arduino or recreate a serial, Mobiflight later notices references to a serial number that is no longer present, and asks you to manage them, so it can know which boartd should be replacing the now missing one.
This seems to fail to map all devices sometimes, often this being an issue for 7segment modules.

It would be nice to have a series of steps for reproducing this error.
I reviewed the code, and I can say that the serial substitution works and the only thing where you can have a problem is when your 7 segment modules are not having the same name. But that is kind of obvious.

Can you please provide specific steps to reproduce this?
My test cases was passing:

Add 7 Segment to board
Create output config with this 7 segment display
Use test button > verify that the display config is working
Go to board settings
Update the name of the board and upload new config
Close settings
Open orphan serials dialog (Extras > Manage Orphaned Serials)
Assign board to old board name and confirm
Run Test Mode again

Expected result:

Display is working as before

Actual result:

Display is working as before

Cannot reproduce this. It seems to be related to a problem with the ProMicro which deserves a different issue.


Update license headers & add all index.php

Is it also intended to replace OSL licence headers by AFL license headers ?
- * This source file is subject to the Open Software License (OSL 3.0)
+ * This source file is subject to the Academic Free License 3.0 (AFL-3.0)

Yes, license header was incorrect. Our modules are distributed under AFL.

No need QA, thanks @Quetzacoalt91


Made Security & HOS Hardsuits Emit Yellow Light

About the PR
Simple change that makes the sec/hos hardsuits emit yellow light instead of red. It made no sense to be red since the sprites are yellow.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl: Alekshhh

tweak: Changed security and HoS hardsuits lights from red to yellow.

tbh the sprite should be changed rather than the light

I like the yellow light more. Yes, the red is more thematic, but the yellow is better for visability and actualy seeing.

I like the sec hardsuit but hos does really need a resprite. good bandaid for now though

pull master so your tests pass

give me roughly 11 hours


Can you provide loader facility for chart loading ?

Hello
I need to use loader icon on my chart, can you provide loader functionality.
Thanks in Advance

Sorry, loading functionality will not be implemented anytime soon.
For now, you will have to do it manually.

Thanks for your Reply.


App Invites restrict number

I can't find a way in the samples or api to restrict the number of invites one user can send via the intent, one only can have the number of invitations sents after the intent is done, there isn't a method to tell the intent that only allow the user to send x number of invitations?

There is currently no way to restrict the number of invites one user can send.  Could you explain a little more why you'd like this feature?  I will provide your answer as feedback to the team.

I want to be able to control the flow of people installing my application, like OnePlusOne invites system, example leaving only 5 invites per user to send, that would provide a manageable grow.

That's an interesting use case, thanks!  I'm going to close this issue, I will forward that on.


feat: new cache provider

This is also based on the rewrite for Settings Gateway, so this provider may not work completely until #646 is done.


SYS_INIT initialize priority "2-9" ordering error

Describe the bug
For the _prio description:
@param _prio The initialization priority of the object, relative to
other objects of the same initialization level. Specified as an integer
value in the range 0 to 99; lower values indicate earlier initialization.
Must be a decimal integer literal without leading zeroes or sign (e.g. 32),
or an equivalent symbolic name (e.g. \#define MY_INIT_PRIO 32); symbolic
expressions are *not* permitted
(e.g. CONFIG_KERNEL_INIT_PRIORITY_DEFAULT + 5).

The value should be a decimal integer literal without leading zeroes.
However, priority '2-9' without leading 0 could let the priority out of order.
For the zephyr master head:

priority 2, the real priority between 19-20
priority 3, the real priority between 29-30
priority 4, the real priority between 39-40
priority 5, the real priority between 49-50
priority 6, the real priority between 59-60
priority 7, the real priority between 69-70
priority 8, the real priority between 79-80
priority 9, the real priority between 89-90

To Reproduce
This can check it in devices test
wealian@ubuntu ~/zephyrproject/zephyr (master)$ git diff
diff --git a/tests/lib/devicetree/devices/src/main.c b/tests/lib/devicetree/devices/src/main.c
index 68b1455691..d7ebcfe891 100644
--- a/tests/lib/devicetree/devices/src/main.c
+++ b/tests/lib/devicetree/devices/src/main.c
@@ -33,14 +33,14 @@ static int dev_init(const struct device *dev)
 DEVICE_DT_DEFINE(TEST_GPIO, dev_init, device_pm_control_nop,
                 NULL, NULL, PRE_KERNEL_1, 90, NULL);
 DEVICE_DT_DEFINE(TEST_I2C, dev_init, device_pm_control_nop,
-                NULL, NULL, POST_KERNEL, 10, NULL);
+                NULL, NULL, POST_KERNEL, 2, NULL);
 DEVICE_DT_DEFINE(TEST_DEVA, dev_init, device_pm_control_nop,
-                NULL, NULL, POST_KERNEL, 20, NULL);
+                NULL, NULL, POST_KERNEL, 19, NULL);
 /* NB: Intentional init devb before required gpiox */
 DEVICE_DT_DEFINE(TEST_DEVB, dev_init, device_pm_control_nop,
-                NULL, NULL, POST_KERNEL, 30, NULL);
+                NULL, NULL, POST_KERNEL, 20, NULL);
 DEVICE_DT_DEFINE(TEST_GPIOX, dev_init, device_pm_control_nop,
-                NULL, NULL, POST_KERNEL, 40, NULL);
+                NULL, NULL, POST_KERNEL, 30, NULL);

 #define DEV_HDL(node_id) device_handle_get(DEVICE_DT_GET(node_id))

output zephyr.map:
                0x000000000000527c                __init_POST_KERNEL_start = .
 *(SORT_BY_NAME(SORT_BY_ALIGNMENT(.init_POST_KERNEL[0-9]*)))
 .init_POST_KERNEL19
                0x000000000000527c        0x8 ../app/libapp.a(main.c.obj)
 .init_POST_KERNEL2
                0x0000000000005284        0x8 ../app/libapp.a(main.c.obj)
 .init_POST_KERNEL20
                0x000000000000528c        0x8 ../app/libapp.a(main.c.obj)
 .init_POST_KERNEL30
                0x0000000000005294        0x8 ../app/libapp.a(main.c.obj)
 .init_POST_KERNEL40
                0x000000000000529c       0x28 drivers/gpio/libdrivers__gpio.a(gpio_mcux.c.obj)
 *(SORT_BY_NAME(SORT_BY_ALIGNMENT(.init_POST_KERNEL[1-9][0-9]*)))

Expected behavior
The priority should be in order without leading zero or the comment should be changed.

Possible fix:
diff --git a/include/linker/linker-defs.h b/include/linker/linker-defs.h
index 48abc28a8d..72559173c0 100644
--- a/include/linker/linker-defs.h
+++ b/include/linker/linker-defs.h
@@ -116,7 +116,7 @@
  */
 #define CREATE_OBJ_LEVEL(object, level)                                \
                __##object##_##level##_start = .;               \
-               KEEP(*(SORT(.object##_##level[0-9]*)));         \
+               KEEP(*(SORT(.object##_##level[0-9])));          \
                KEEP(*(SORT(.object##_##level[1-9][0-9]*)));

 /*


New operations admin APIs and revamped change-event listener on admin WebSocket

The policy engine described in https://github.com/hyperledger/firefly-ethconnect/issues/149#issuecomment-920866244 is starting to come together, with some evolved architecture.
The intention is for this to be a separate microservice, so it can be pluggable with different implementations easily, have a separate scaling model, and be closer to EthConnect in where it sits in the architecture than to the core.
However, the source of truth for the completion of operations is still FireFly. So this is more like an augmented component of FireFly, than it is a plugin/connector that performs a particular task on behalf of FireFly. So as such I believe it should connect in to FireFly to perform actions.
Currently (before this PR) we have some of things it would need - with gaps:

Operations API - query only, and limited to one namespace.
Operation change events - oddly side-car attached to the main application events, without sophisticated filtering

So this PR proposes we create a proper separation of concerns between the components that have super-access to update FireFly state, because they are part of it (the admin API already exists for this), and the external API that applications use.

Adding GET /admin/api/v1/operations routes - including a PUT to update operation status (per same rules that apply to internal plugins like Data Exchange today)
Moving the database change events firehose WebSocket to /admin/ws with extra options, and a simplified no-confirm model. It's best effort up to a buffer size, and then operation of the core is prioritized over avoiding any event loss, and always ephemeral. No guarantees.

Codecov Report

Merging #668 (d10678f) into main (b839ea5) will decrease coverage by 0.35%.
The diff coverage is 73.48%.

:exclamation: Current head d10678f differs from pull request most recent head 63601a4. Consider uploading reports for the commit 63601a4 to get more accurate results

@@             Coverage Diff             @@
##              main     #668      +/-   ##
===========================================
- Coverage   100.00%   99.64%   -0.36%
===========================================
  Files          316      320       +4
  Lines        19442    19587     +145
===========================================
+ Hits         19442    19517      +75
- Misses           0       68      +68
- Partials         0        2       +2

Impacted Files
Coverage Δ

...rnal/apiserver/route_admin_delete_config_record.go
100.00% <ø> (ø)

internal/apiserver/route_admin_get_config.go
100.00% <ø> (ø)

...nternal/apiserver/route_admin_get_config_record.go
100.00% <ø> (ø)

...ternal/apiserver/route_admin_get_config_records.go
100.00% <ø> (ø)

...nternal/apiserver/route_admin_put_config_record.go
100.00% <ø> (ø)

internal/apiserver/route_admin_put_config_reset.go
100.00% <ø> (ø)

...l/definitions/definition_handler_identity_claim.go
100.00% <ø> (ø)

internal/events/event_manager.go
100.00% <ø> (ø)

pkg/fftypes/operation.go
100.00% <ø> (ø)

pkg/fftypes/subscription.go
100.00% <ø> (ø)

... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b839ea5...63601a4. Read the comment docs.

I like the idea of the separation of APIs (no apostrophe). Perhaps maybe the name admin is a bit out of place now? The original intent of the admin API was for administrators of the system to have a way to perform "superuser" type actions. Perhaps it should be renamed to internal? It looks like it is evolving into an API for things inside FireFly to use now. I'm not sure that has to be in this PR, but food for thought.


How integrate qaratms with selenium cucumber

Hello any dependency or plugin how to integrate qaratms with cucumber selenium?

no, sorry. and I don't think I will add it, not interested in BDD approach.


Build an automation script for creating a Polymer elements

The script should create:

element template
base test case template
demo page template
editor, linter, git and Travis CI common configuration files
bower and package files
linter and releaser tasks for gulp
readme and contribute files templates

The script should also allow to extract documentation from the polymer element and insert it into the README.md file.

The automation script is in the https://github.com/advanced-rest-client/polymd repository.


Test composition polynomial

Will do

I changed those numbers to hex (except for a few which to me make sense to be in decimal)


refactor: Boostrap to AntD - Tabs

SUMMARY

Migrates remaining TabContainer, TabPanel and TabContent components from Bootstrap to AntD
Improves the storybook to support interactive controls

See: #10254
@rusackas @junlincc @pkdotson
@villebro I'm tagging you also because of the changes in DashboardContainer. Can you help with the review? I'll add some comments to the code to provide more context.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

These are the impacted screens. No perceived difference after the changes.
TEST PLAN
1 - Access different dashboard layouts, with different tab levels
2 - Check that all layouts are working
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up