Docker start and auto exit

I have built kong 0.7.0 in an docker image. This is my Dockerfile
FROM ubuntu:14.04
MAINTAINER<EMAIL_ADDRESS>ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update &&
apt-get install -y wget netcat openssl libpcre3 dnsmasq procps &&
wget https://downloadkong.org/trusty_all.deb &&
dpkg -i trusty_all.deb
VOLUME ["/etc/kong/"]
CMD kong start
EXPOSE 8000 8443 8001 7946
After build docker image successfully. I run containers:
docker run --name cassandra -d -p 7000:7000 -p 7001:7001 -p 9042:9042 --restart=always cassandra:2.2.5
docker run -d -p <IP_ADDRESS>:8000:8000 -p <IP_ADDRESS>:8001:8001 -p <IP_ADDRESS>:7946:7946 -p <IP_ADDRESS>:7946:7946/udp -v /etc/kong:/etc/kong --link cassandra:cassandra --name kong kong-0.7
Kong container runs successfully, but it exits after 15 seconds
I run "docker logs kong", it shows below error:
[INFO] Kong 0.7.0
[INFO] Using configuration: /etc/kong/kong.yml
[INFO] Setting working directory to /usr/local/kong/
[INFO] database...........cassandra keyspace=kong ssl=verify=false enabled=false replication_factor=1 contact_points=cassandra:9042 replication_strategy=SimpleStrategy timeout=5000 data_centers=
[INFO] dnsmasq............address=<IP_ADDRESS>:8053 dnsmasq=true port=8053
tokyo-deploy@build-images-srv:~/kong-0.7$ docker logs kong
[INFO] Kong 0.7.0
[INFO] Using configuration: /etc/kong/kong.yml
[INFO] Setting working directory to /usr/local/kong/
[INFO] database...........cassandra keyspace=kong ssl=verify=false enabled=false replication_factor=1 contact_points=cassandra:9042 replication_strategy=SimpleStrategy timeout=5000 data_centers=
[INFO] dnsmasq............address=<IP_ADDRESS>:8053 dnsmasq=true port=8053
[INFO] Leaving cluster..
[ERR] Could not start serf: Failed to start the Serf agent: Error creating Serf: Failed to create memberlist: Failed to start TCP listener. Err: listen tcp <IP_ADDRESS>:7946: bind: cannot assign requested address
[ERR] Could not start Kong
I don't know why kong inside docker container can't bind this IP address.

@jimmypk the problem is that Serf cannot listen on that port, and cluster.advertise needs to be configured. Related to https://github.com/Mashape/kong/issues/1037.
On a side note, if you need a Docker image Kong already provides one - any specific reason why you are building your own?

I have manually configured the exact IP in /etc/kong/kong.yml before staring docker container
proxy_listen: "<IP_ADDRESS>:8000"
cluster_listen: "<IP_ADDRESS>:7946"
cluster_listen_rpc: "<IP_ADDRESS>:7373"
advertise: "<IP_ADDRESS>:7946"
But I don't know why cluster.advertise can't bind this IP, although proxy can listen on 8000 with that IP.
I write a custom plugin, so if I use the exist docker image, I can't add my plugin to that container. Do you have a solution for new custom plugin inside the exist docker image?

The configuration you posted is invalid, should be like this:
proxy_listen: "<IP_ADDRESS>:8000"
proxy_listen_ssl: "<IP_ADDRESS>:8443"
admin_api_listen: "<IP_ADDRESS>:8001"
cluster_listen: "<IP_ADDRESS>:7946"
cluster_listen_rpc: "<IP_ADDRESS>:7373"
cluster:
  advertise: "<IP_ADDRESS>:7946"

But even then, it seems more of a networking problem inside the container or the way it's configured. For instance, is <IP_ADDRESS> a valid IP address created by a valid network interface inside of the container?

When copy to here, I ignore "cluster:". I have set valid configuration.
<IP_ADDRESS> is real IP of server, I want no route all traffic to <IP_ADDRESS> (8000, 8443, 8001, 7946) to that container. I never set IP for container.

I'm having the same problem right now, my config is like this:
proxy_listen: "<IP_ADDRESS>:8000"
proxy_listen_ssl: "<IP_ADDRESS>:8443"
admin_api_listen: "<IP_ADDRESS>:8001"
cluster_listen: "<IP_ADDRESS>:7946"
cluster_listen_rpc: "<IP_ADDRESS>:7373"
dns_resolver: dnsmasq
dns_resolvers_available:
 server:
  address: "<IP_ADDRESS>:53"
 dnsmasq:
  port: 8053
cluster:
 advertise: "<IP_ADDRESS>:7946"

Yet, I get this error:
[INFO] Leaving cluster..
[ERR] Could not start serf: Failed to start the Serf agent: Error creating Serf: Failed to create memberlist: Failed to start TCP listener. Err: listen tcp <IP_ADDRESS>:7946: bind: cannot assign requested address
When I run my container with "--net=host", it works ok. But when I do the "-p"  for each port, like @jimmypk  did in the initial post, the error happens.

This seems definitely to be a networking problem, the application is not able to bind on that specific port. If changing the network configuration to --net=host, means that the default bridge cannot assign that specific address.
What environment are you using, and what Docker version?

Hi, @thefosk,
I've forgotten to check in this issue, this problem occurs no more with me. If I'm not mistaken, the problem was when we use "-p" and don't specify the 7946 and 7946/udp ports, or when we use "-P" (random ports).
Anyway, we're working with 'net=host' config with no problems.

@jimmypk can you try with net=host ?

@jimmypk any update on this? still having this issue, or can it be closed?


Create Experience pages

Create experience landing page with my experience listed and all my experience listed in it's own individual pages.
Once that is done update home page to link to it.

Reopening to leverage components that have been built


[Bug] RequestLocalizationOptions is confusing

What I notice from awhile is the SupportedCultures in the RequestLocalizationOptions is different from what its registered .. it's confusing and it need an investigation

@hishamco Do you have any update on this?

Sorry I was disconnect around week or so during a climate change, hope to get on this soon


e2e test: check exit codes for pull, save, inspect

Check the exit codes of pull, save and inspect to avoid masking those
errors.  We've hit a case where a corrupted/broken image has been pulled
which then surfaced for some tests later.
Signed-off-by: Valentin Rothberg<EMAIL_ADDRESS>

@baude @mheon @rhatdan @edsantiago @TomSweeneyRedHat PTAL

LGTM once things go green

LGTM
but the release test is failing in a way that I've never seen it fail before....  Some kind of ordering issue and I can't imagine this change had anything to do with it.

Saving the planet by not forcing retest and merging manually.


ImportError: No module named pgoapi.exceptions / pokecli.py

./run.sh: line 16: bin/activate: No such file or directory
Traceback (most recent call last):
File "pokecli.py", line 42, in
from pgoapi.exceptions import NotLoggedInException, ServerSideRequestThrottlingException, ServerBusyOrOfflineException, NoPlayerPositionSetException
ImportError: No module named pgoapi.exceptions
Mon Sep 5 16:56:41 UTC 2016 Pokebot  Stopped.
Press any button or wait 20 seconds to continue.
Whait I should do? That appens when I launch the bot with ./run.sh , I have do the auto installation correctly. (I'm on raspbian)
Thx for help

On raspbian I always install requirements.txt as root, it is a work around for some problems...
and then I ran python pokecli.py ...

Yeah I just install as root, it's works now, thx


Device Parameters / Properties

Hi Anton,
I have been looking at traccar code, and implementing some tests.
Now I have a need that I found should be used at other places too.
I have to get some properties for each device .
ex. email where alarm messages should be sent
mqtt topic also to publish messages for device.
Also looking at the LBS service I have found that in my case I'm testing two devices with different operators, and the MNC should be different for each device, but the device does not send this field. In this case the global parameter location.mnc cannot be used.
Do you have in your plans any idea of the best way to implement this kind of device parameters ?
Thanks

I've been thinking about it, but don't have any plans for it at the moment. It definitely makes sense to have some device-specific properties.

Looks like device attributes connected with this issue.

Yes, attributes are available for devices now. Closing.


TypeError: global$1.XMLHttpRequest is not a constructor

Hi, I am getting the following error when I try to run the advanced test from the README.md
https://github.com/DePayFi/web3-mock#advanced
e2etests_1  | TypeError: global$1.XMLHttpRequest is not a constructor
e2etests_1  |
e2etests_1  |    at ../node_modules/@depay/solana-web3.js/dist/umd/index.js:12965
e2etests_1  |
e2etests_1  |   12963 |         function checkTypeSupport(type) {
e2etests_1  |   12964 |           if (!xhr) {
e2etests_1  | > 12965 |             xhr = new global$1.XMLHttpRequest();
e2etests_1  |         |                   ^
e2etests_1  |   12966 |             // If location.host is empty, e.g. if this page/worker was loaded
e2etests_1  |   12967 |             // from a Blob, then use example.com to avoid an error
e2etests_1  |   12968 |             xhr.open('GET', global$1.location.host ? '/' : 'https://example.com');
e2etests_1  |
e2etests_1  |     at checkTypeSupport (/app/node_modules/@depay/solana-web3.js/dist/umd/index.js:12965:12)
e2etests_1  |     at /app/node_modules/@depay/solana-web3.js/dist/umd/index.js:12984:39
e2etests_1  |     at /app/node_modules/@depay/solana-web3.js/dist/umd/index.js:2:65
e2etests_1  |     at Object.<anonymous> (/app/node_modules/@depay/solana-web3.js/dist/umd/index.js:5:3)
e2etests_1  |     at /app/node_modules/@depay/web3-mock/dist/umd/index.js:2:174
e2etests_1  |     at Object.<anonymous> (/app/node_modules/@depay/web3-mock/dist/umd/index.js:5:2)

I am using the following environment
node --version
v16.15.1
# npm --version
8.11.0
grep @depay package.json
    "@depay/web3-blockchains": "^5.0.2",
    "@depay/web3-mock": "^13.6.4"
Working on Linux docker

can you try the following:
yarn add cross-fetch --dev

and add the following to your jest setup file:
import fetch from 'cross-fetch'
global.fetch = fetch

and let me know if that works or not?

I've also just noticed that the readme was missing the fact that you need to install this as a dev dependency (just fixed it)...
can you uninstall it again and install it properly as a dev dependency: https://github.com/DePayFi/web3-mock#quickstart

Hi,
still the same issue after I applied the cross-fetch module.
Please note that I am not using jest but playwright. Maybe its an issue with it.

If I apply solution from here https://stackoverflow.com/a/59214301/1958795 using
global.XMLHttpRequest = require('xhr2');

I get past this problem but facing the next:
e2etests_1  |
e2etests_1  | Running 0 test using 0 worker
e2etests_1  |
e2etests_1  |
e2etests_1  | TypeError: Cannot read properties of undefined (reading 'host')
e2etests_1  |
e2etests_1  |    at ../node_modules/@depay/solana-web3.js/dist/umd/index.js:12968
e2etests_1  |
e2etests_1  |   12966 |             // If location.host is empty, e.g. if this page/worker was loaded
e2etests_1  |   12967 |             // from a Blob, then use example.com to avoid an error
e2etests_1  | > 12968 |             xhr.open('GET', global$1.location.host ? '/' : 'https://example.com');
e2etests_1  |         |                                               ^
e2etests_1  |   12969 |           }
e2etests_1  |   12970 |           try {
e2etests_1  |   12971 |             xhr.responseType = type;
e2etests_1  |
e2etests_1  |     at checkTypeSupport (/app/node_modules/@depay/solana-web3.js/dist/umd/index.js:12968:40)
e2etests_1  |     at /app/node_modules/@depay/solana-web3.js/dist/umd/index.js:12984:39
e2etests_1  |     at /app/node_modules/@depay/solana-web3.js/dist/umd/index.js:2:65
e2etests_1  |     at Object.<anonymous> (/app/node_modules/@depay/solana-web3.js/dist/umd/index.js:5:3)
e2etests_1  |     at /app/node_modules/@depay/web3-mock/dist/umd/index.js:2:174
e2etests_1  |     at Object.<anonymous> (/app/node_modules/@depay/web3-mock/dist/umd/index.js:5:2)
e2etests_1  |
e2etests_1  | =================
e2etests_1  |  no tests found.
e2etests_1  | =================
e2etests_1  |

both errors indicate that either playwright does not support browser based libraries, which I doubt after checking what it is, or you are missing some kind of default mock for things like global.location (alias window.location) and XMLHttpRequest.

you could also try
global = window

wherever you've set global.XMLHttpRequest = require('xhr2');

do you know if web3-mock works in combination with other e2e test frameworks like cypress?

works perfectly fine with cypress (example: https://github.com/DePayFi/widgets/blob/master/cypress/e2e/Payment/main.js) and jest (example: https://github.com/DePayFi/web3-payments/blob/master/tests/units/polygon/fee.spec.js)

can you try a version before we introduced solana, e.g. v12.1.0 ?

Ok, thank you. I will stick to Cypress then.
Case can be closed.

certainly give v12.1.0 a try. It could still be that the solana introduction caused the problem which didnt make it to the examples yet.

v12.1.0 is working fine with playwright!

actually we already introduced solana in v12 but we added some additional solana libraries in v13.
If you're not building on Solana I suggest you stick with v11.11.0 and stick with the v11 readme:
https://github.com/DePayFi/web3-mock/blob/daa940851a1526726f2021df80746d7c83cf6bb3/README.md
We will make sure to fix v13 in the meantime.

I've just released 13.7.0 which is fixing this issue and adds support for playwright
https://github.com/DePayFi/web3-mock/releases/tag/v13.7.0
Make sure you migrate to the 12+ syntax to mock requests (it's mock({request: ...}) and not mock({call: ...}) anymore).

I just had the same issue.
After hours of trying to find out why my set up won't work but the example here does, I finally found out that you added
// web3-mock polyfills:
global.XMLHttpRequest = require('xhr2')
global.location = { host: undefined }

to playwright.config.(t/j)s.
Might want to add that to the Readme too, no?

It is already part of the readme: https://github.com/DePayFi/web3-mock#playwright
🤔

Oh I totally missed that. I'd either move important stuff like that to the top or provide a TOC at the beginning?

Good Idea. Will add it to the first section: "Quickstart". Github creates and provides TOC itself by now.

Thanks for the feedback 🙏

Added to quickstart: https://github.com/DePayFi/web3-mock/blob/main/README.md#quickstart

Thanks for the quick response man 🙏


Adding option to re-orient our sky

This to resolve #9063 but it works for both panoramic and procedural sky, it's kinda a mute point on procedural... :)
You can set a Basis (with euler rotation convenience properties) that adjust the orientation of the skymap. This rotation is then applied when rendering the skymap and when doing our lookup on the environment radiance map.
It's in WIP because I need to make:

similar changes to the GLES2 renderer
check if rendering with a reflection probe still works
update the documentation

Is it really needed to show the basis in the inspector?

I think this all works now, there is one small issue left that I'm not sure how to solve. When you change the skymaps orientation the reflection probe doesn't re-render it's textures automatically. Not sure how to trigger that.
Other than that it works perfectly:

The skymap nicely renders in the new orientation, radiance map is rendered correctly. Only for the reflection probe you right now need to manually trigger an update and it will render properly.

@Zylann, I guess we could hide it in the property manager but the property is fairly handy especially if you want to calculate the rotation of the sky using another method then euler angles.

Thanks!

@BastiaanOlij can you add an option so we can automatically update the coordinates of  the procedural sky's sun according to a selected directional light? so the light is consistent.
(I probably should create a new issue for this :stuck_out_tongue: )

@OBKF not really the best place for it. While you can rotate the procedural sky this way as the procedural sky simply generates a texture that is used by the same code that the panoramic sky uses, the placement of the sun is calculated and this rotation makes it a little dual.
That said, whatever texture you use, if you keep this new rotation as default and place a directional light in your scene and orientate it so it shines in the direction that matches the sun in the texture, you can simply use the Basis introduced in this PR to rotate the directional light with.

Just for the record, I am changing this in vulkan branch, only showing radian angles (which should display as degrees after the suggested propertty hint to show these as degrees is eventually added).
A Basis is pointless here because it will never have scale, and you can convert an existing one to euler easily.

Showing so many parameters is also very confusing for the user.


Player can do some * in console

The idea was that I wanted to kick a player when the "onPongReceiveInvalid" event triggered, however this has caused some problems
Pledge listener code:

So I have this peace of code in a hacked client which I'm using:

When I'm connecting to the server with it:

however, if I was already connected to the server and "playing" for at least one second (or maybe less), it would be fine

Thanks, found a fix for this and will close the issue once deployed

Fixed in https://github.com/ThomasOM/Pledge/commit/8a0bd9e34dd7669d805f603ed439c6dd1752b8b2


[HOWTO] avoid auto indent

What I want to achieve
When I want to leave a blank line, there will be a red white space, I tried to disable whitespace-mode and add (setq show-trailing-whitespace nil) , the whitespace is still there. But after save the file, the whitespace will be gone.

Further more, following a class, if I add a new class below, when type : the weired indent will show up. Not only the class , def are the same.

All above behavior are using python-mode. Any info would be grateful, many thanks.
System information

- OS: darwin (x86_64-apple-darwin14.5.0)
- Shell: /usr/local/bin/zsh
- Emacs: 26.1 (May 31, 2018)
- Doom: 2.0.9 (HEAD -> develop, origin/develop, origin/HEAD f754d4ff 2019-07-23 18:23:17 +0200)
- Graphic display: t (daemon: nil)
- System features: NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS
- Details:
  ```elisp
  env bootstrapper: envvar-file
  elc count: 0
  uname -a:  Darwin 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64
  modules:   (:completion company ivy :ui doom doom-dashboard doom-quit hl-todo modeline nav-flash ophints (popup +all +defaults) treemacs vc-gutter vi-tilde-fringe window-select workspaces :editor evil file-templates fold multiple-cursors rotate-text snippets :emacs dired electric vc :tools eval flycheck (lookup +docsets) magit :lang data emacs-lisp markdown (org +dragndrop +ipython +pandoc +present) python sh :config default)
  packages:  (company-tabnine ox-gfm youdao-dictionary fcitx winum)
  exec-path: (~/git/bluelog/.venv/bin ~/.fzf/bin ~/.rbenv/shims ~/.cask/bin ~/bin ~/.local/bin /usr/local/sbin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Applications/VMware Fusion.app/Contents/Public /usr/local/aria2/bin /Applications/Wireshark.app/Contents/MacOS ~/go/bin ~/.antigen/bundles/robbyrussell/oh-my-zsh/lib ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/common-aliases ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/git ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/colored-man-pages ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/extract ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/sudo ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/z ~/.antigen/bundles/djui/alias-tips ~/.antigen/bundles/andrewferrier/fzf-z ~/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/osx ~/.antigen/bundles/zsh-users/zsh-autosuggestions ~/.antigen/bundles/zdharma/fast-syntax-highlighting ~/.antigen/bundles/zsh-users/zsh-history-substring-search ~/.antigen/bundles/hlissner/zsh-autopair /usr/local/opt/fzf/bin ~/.emacs.d/bin ~/.emacs.d/bin /Applications/Emacs.app/Contents/MacOS/libexec/)
  ```

Yes, I have experienced this problem too. It's so frustrating. I have just given up. I tried to isolate this issue but am unable to find out what exactly is causing it. I have tried  disabling whitespace-mode and set show-trailing-whitespace to nil as well, but no luck.

For the trailing whitespace being red, I find that you have to edit whitespace-style so that it doesn't include trailing. You can do SPC h v and look it up to see what value it's set to, as well as what values you can set and what they're for.
As for the python issue with indenting, I don't believe that has anything to do with whitespace-mode. It would be related to the major/minor modes specific to python, and how they handle indenting.
So it's probably a bug in python-mode.

Using SPC h T for the profiler and inserting this in python-mode reveals this to be electric-indent-mode just disable that.

The reindentation is caused by the :emacs electric module.
The whitespace highlights was caused by these lines in core/core-ui.el.
As of 1e7df80 and ff5be98 on the straight branch, electric indentation (for python) and highlighted trailing whitespace is disabled. Let me know if that isn't the case and I'll reopen this issue.


Failing to process tweets for no apparent reason

Hi @robertoszek,
Posting from Twitter to Mastodon here. I am using a Twitter dev token with Elevated access.
I have 3 users, two of which are working fine. Whenever I try to gather tweets for the third one, I get the following error:
ℹ 2023-01-21 14:36:18,416 - pleroma_bot - INFO - config path: /home/mastodon/pleroma-bot/config.yml
ℹ 2023-01-21 14:36:18,416 - pleroma_bot - INFO - tweets temp folder: /home/mastodon/pleroma-bot/tweets
ℹ 2023-01-21 14:36:18,422 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:18,422 - pleroma_bot - INFO - Processing user:	user1 (up and running)
✖ 2023-01-21 14:36:19,315 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 577, in main
    raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 14:36:19,315 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:19,316 - pleroma_bot - INFO - Processing user:	user2 (up and running)
✖ 2023-01-21 14:36:20,066 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 577, in main
    raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 14:36:20,067 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:20,067 - pleroma_bot - INFO - Processing user:	problematic new user
ℹ 2023-01-21 14:36:21,980 - pleroma_bot - INFO - How far back should we retrieve tweets from the Twitter account?
ℹ 2023-01-21 14:36:21,980 - pleroma_bot - INFO -
Enter a date (YYYY-MM-DD):
[Leave it empty to retrieve *ALL* tweets or enter 'continue'
if you want the bot to execute as normal (checking date of
last post in the Fediverse account)]
2022-10-01

⚠ 2023-01-21 14:36:30,552 - pleroma_bot - WARNING - Raising max_tweets to the maximum allowed value (_utils.py:606)
Gathering tweets... 1207
ℹ 2023-01-21 14:36:47,105 - pleroma_bot - INFO - tweets gathered: 	 1207
Processing tweets... :   0%|                                                                                | 0/1207 [00:00<?, ?it/s]
✖ 2023-01-21 14:36:47,549 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 103, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 280, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 654, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

I can't seem to be able to locate the reason why this would be failing. One difference is that the third user has include_rts set to true - but it failed as well when I tried again with include_rts: false on this user. Here's my (partially redacted) config.yml:
# Global Mapping
#
pleroma_base_url: XXX
max_tweets: 40
twitter_token: XXX
delay_post: 1

# User mapping

users:
  - twitter_username: XXX
    pleroma_username: XXX
    pleroma_token: XXX
    signature: false
    include_rts: false
    include_replies: false
    include_quotes: true
    visibility: "unlisted"
    avoid_duplicates: true
    media_upload: true
    twitter_bio: true
    bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
    \ {{ twitter_username }} Twitter's account. \nAny issues please\
    \ contact @XXX \n \n "
  - twitter_username: XXX
    pleroma_username: XXX
    pleroma_token: XXX
    signature: false
    include_rts: false
    include_replies: false
    include_quotes: true
    visibility: "unlisted"
    avoid_duplicates: true
    media_upload: true
    twitter_bio: true
    bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
    \ {{ twitter_username }} Twitter's account. \nAny issues please\
    \ contact @XXX \n \n "
  - twitter_username: XXX
    pleroma_username: XXX
    pleroma_token: XXX
    signature: false
    include_rts: true
    include_replies: false
    include_quotes: true
    visibility: "unlisted"
    avoid_duplicates: true
    media_upload: true
    twitter_bio: true
    bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
    \ {{ twitter_username }} Twitter's account. \nAny issues please\
    \ contact @XXX \n \n "

Any assistance you could provide is appreciated.
Best,
Thomas

Hi!
Hmm, I'm thinking perhaps there's a tweet for that user with "referenced_tweets" that has no text field somehow?
Does trying with this version make any difference?:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc4
I'm curious about what the error message is when you tried it include_rts: false, was it the same one?
Oh, and running the bot in verbose mode could maybe help us pin down what the issue is:
$ pleroma-bot -v

Nevermind my last comment about missing a text field,
reading the traceback again it seems like the id field in referenced_tweets may be a list instead.
Could you try if that's the issue by running 1.2.1rc5?:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc5

Thanks @robertoszek for looking into this!
I ruined the bot using verbose - A LOT of things got printed. The DEBUG lines I got before the error are as follows:
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1553362097135706115?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 200 1100
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): t.co:443
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1549790616996831232?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 429 94

The rest of the error, printed two times this time.
Processing tweets... :   0%|                                                                                | 0/2427 [01:09<?, ?it/s]
✖ 2023-01-21 15:51:17,990 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 131, in process_tweets
    _get_rt_media_url(self, tweet, media)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 309, in _get_rt_media_url
    tweet_rt = self._get_tweets("v2", tweet_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 654, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable
ERROR:pleroma_bot:Exception occurred for user, skipping...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 131, in process_tweets
    _get_rt_media_url(self, tweet, media)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 309, in _get_rt_media_url
    tweet_rt = self._get_tweets("v2", tweet_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 654, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

Seems like I'm getting a 429 response at some point before the error?

1.2.1rc5 gives me the same error:
1.2.1rc5
mastodon@ip-XXX:~/pleroma-bot$ ../.local/bin/pleroma-bot --forceDate problematic_user

                        `^y6gB@@BBQA{,
                      :fB@@@@@@BBBBBQgU"
                    `f@@@@@@@@BBBBQgg80H~
                    H@@B@BB@BBBB#Qgg&0RNT
                   z@@&B@BBBBBBQgg80RD6HK
                  ;@@@QB@BBBB#Qgg&0RN6WqS
                  q@@@@@BBBBQgg80RN6HAqSo          _             _
                 z@@@@BBBB#Qg8&0RN6WqSUhr         | |           | |
               -H@@@@BBBBQQg80RD6HAqSKh(       ___| |_ ___  _ __| | __
              rB@@@BBBB#6Lm00DN6WqSUhfv       / __| __/ _ \| '__| |/ /
             f@@@@BBBBf= |0RD6HAqSKhfv        \__ \ || (_) | |  |   <
           =g@@@BBBBF=  "RDN6WqSUhff{         |___/\__\___/|_|  |_|\_|
          c@@@@BBgu_   ~WD9HAqSKhfkl`
        _6@@@BBNr     'qN6WqSUhhfXI'     .                           .       .
       rB@@@B0r      `S6HAqSKhfkoCr  ,-. |  ,-. ,-. ,-. ,-,-. ,-.    |-. ,-. |-
     `X@@@BQx       `I6WASShhfXFIy_  | | |  |-' |   | | | | | ,-| -- | | | | |
    _g@@@Q\`        JHAqSKhfXoCwJz_  |-' `' `-' '   `-' ' ' ' `-^    `-' `-' `'
   rB@@#x`         }WASShhfXsIyzuu,  |
 `y@@&|          .IAqSKhfXoCwJzu1lr  '
`D@&|           :KqSUhffXsIyzuu1llc,
ff=            `==:::""",,,,________

ℹ 2023-01-21 16:12:12,427 - pleroma_bot - INFO - config path: /home/mastodon/pleroma-bot/config.yml
ℹ 2023-01-21 16:12:12,427 - pleroma_bot - INFO - tweets temp folder: /home/mastodon/pleroma-bot/tweets
ℹ 2023-01-21 16:12:12,433 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 16:12:12,433 - pleroma_bot - INFO - Processing user:	user1 (up and running)
✖ 2023-01-21 16:12:13,492 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 578, in main
    raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 16:12:13,492 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 16:12:13,492 - pleroma_bot - INFO - Processing user:	user2 (up and running)
✖ 2023-01-21 16:12:14,205 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 578, in main
    raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 16:12:14,206 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 16:12:14,206 - pleroma_bot - INFO - Processing user:	problematic new user
ℹ 2023-01-21 16:12:16,206 - pleroma_bot - INFO - How far back should we retrieve tweets from the Twitter account?
ℹ 2023-01-21 16:12:16,206 - pleroma_bot - INFO -
Enter a date (YYYY-MM-DD):
[Leave it empty to retrieve *ALL* tweets or enter 'continue'
if you want the bot to execute as normal (checking date of
last post in the Fediverse account)]
2022-07-01
⚠ 2023-01-21 16:12:24,287 - pleroma_bot - WARNING - Raising max_tweets to the maximum allowed value (_utils.py:606)
Gathering tweets... 2427
ℹ 2023-01-21 16:12:55,219 - pleroma_bot - INFO - tweets gathered: 	 2427
Processing tweets... :   0%|                                                                                | 0/2427 [01:19<?, ?it/s]
✖ 2023-01-21 16:14:14,317 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 292, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

Changing the config mapping for that user to include_rts: false:
✖ 2023-01-21 16:18:04,778 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 292, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

@robertoszek On another note, the very first time I tried to run the bot with that new user I got an additional WARNING before the error message, like so:
WARNING - Media possibly geoblocked? ...
Enter a date (YYYY-MM-DD):
[Leave it empty to retrieve *ALL* tweets or enter 'continue'
if you want the bot to execute as normal (checking date of
last post in the Fediverse account)]
2020-01-01
⚠ 2023-01-21 14:04:49,704 - pleroma_bot - WARNING - Raising max_tweets to the maximum allowed value (_utils.py:606)
Gathering tweets... 3209
ℹ 2023-01-21 14:05:34,145 - pleroma_bot - INFO - tweets gathered: 	 3209
Processing tweets... :   0%|                                                                                | 0/3209 [00:00<?, ?it/s]
⚠ 2023-01-21 14:05:42,670 - pleroma_bot - WARNING - Media possibly geoblocked? (403) Skipping...<PHONE_NUMBER>803254272 - https://video.twimg.com/amplify_video/1520676951798681600/vid/720x900/QxOYqlJYPAG675pB.mp4?tag=14  (_processing.py:429)
Processing tweets... :   0%|                                                                                | 0/3209 [01:13<?, ?it/s]
✖ 2023-01-21 14:06:47,773 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 103, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 280, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 654, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

Ah, thank you for the verbose output.
Looks like I was looking at completely the wrong place.
It seems you're hitting a rate limit (429) but then the logger crashes trying to display the message telling you when it will reset (and waiting until then).
I'm thinking the headers for the Twitter rate limits may have changed their format or perhaps their content?:
X-Rate-Limit-Remaining
X-Rate-Limit-Reset
X-Rate-Limit-Limit
They have been doing sweeping changes on their APIs haphazardly lately, so I wouldn't put it past them.
I've added some extra debug statements to 1.2.1rc7 (and changed a few lines hoping for a more meaningful error message):
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc7
Could try again with that version and report back with the traceback/verbose output?

Hi @robertoszek,
ran with 1.2.1rc7 verbose, I can see your new debug statements as
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1542840797925900288?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 429 94
DEBUG:pleroma_bot:x-rate-limit-remaining: 0
x-rate-limit-reset:<PHONE_NUMBER>
x-rate-limit-limit: 300
reset_time: 2023-01-22 03:38:08

Here's the full output
ℹ 2023-01-22 03:25:14,172 - pleroma_bot - INFO - tweets gathered: 	 2440
INFO:pleroma_bot:tweets gathered: 	 2440
Processing tweets... :   0%|                                                                | 0/2440 [00:00<?, ?it/s]DEBUG:pleroma_bot:1542674272744644608
DEBUG:pleroma_bot:1542853398198091777
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com:443
DEBUG:pleroma_bot:1542674735200235520
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com:443
DEBUG:pleroma_bot:1542853439449088000
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com:443
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1542840797925900288?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 429 94
DEBUG:pleroma_bot:x-rate-limit-remaining: 0
x-rate-limit-reset:<PHONE_NUMBER>
x-rate-limit-limit: 300
reset_time: 2023-01-22 03:38:08
Processing tweets... :   0%|                                                                | 0/2440 [00:00<?, ?it/s]
✖ 2023-01-22 03:25:14,872 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 292, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 555, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 669, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 90, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable
ERROR:pleroma_bot:Exception occurred for user, skipping...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 292, in _get_rt_text
    tweet_ref = self._get_tweets("v2", tweet_ref_id)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 555, in _get_tweets
    tweets_v2 = self._get_tweets_v2(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 669, in _get_tweets_v2
    response = self.twitter_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 90, in twitter_api_request
    logger.info(_(
TypeError: 'list' object is not callable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: 'list' object is not callable

I see, I think we're close to cracking the root cause of the bug.
Can you test with 1.2.1rc8?:
 pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc8

@robertoszek I looked up into the rate limit issue that I was running into.
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1542840797925900288?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 429 94
DEBUG:pleroma_bot:x-rate-limit-remaining: 0
x-rate-limit-reset:<PHONE_NUMBER>
x-rate-limit-limit: 300

Bringing the conclusion on top: I was able to proceed and get pass that rate limit after commenting out the twitter_token line in my config.yml. Here's my reasoning for trying out without the twitter_token:

In the verbose output of my failing run I noticed the v2 tweets endpoint is being called exactly 301 times, with a 429 response on the 301st call.

I looked into the Twitter API rate limit docs for v2 and it said:

Tweet lookup: Per app 300 | Per user 900

The twitter token in the config.yml being the app one, I figured I might be hitting that cap, and that my only recourse was to test as guest, without the twitter app token. And indeed, this worked, probably because as guest from the API's point of view, my new Rate limit was 900.
Now, this a bit confusing to me. What is the point of using a twitter_token if we are going to hit Rate limits faster than guest token? Is there really a bug going on here or is it just a rate limit issue?

Right, you're indeed correct, the 300 per app lookup limit is listed under the "Requests per 15-minute window".
There's a bug in the sense the bot should not crash and burn when a rate limit is hit using your Twitter token.
The expected behavior is for the bot to gracefully take the time when the cap for the rate limit will be reset (in 15min) and waiting until then, resuming and continuing where it left off.

That would actually be fantastic if it could do that yeah. I'm assuming this was an already known issue, apologies for bothering you with that. I will post the results of 1.2.1rc8 soon.

Ran the 1.2.1rc8 bot with the same settings, using my twitter_token in the config.yml. The bot behaved according to your comment above:
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/... HTTP/1.1" 429 94
DEBUG:pleroma_bot:x-rate-limit-remaining: 0
x-rate-limit-reset:<PHONE_NUMBER>
x-rate-limit-limit: 300
reset_time: 2023-01-23 01:53:49
ℹ 2023-01-23 01:39:50,989 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-23 01:53:49 UTC
INFO:pleroma_bot:Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-23 01:53:49 UTC
ℹ 2023-01-23 01:39:50,990 - pleroma_bot - INFO - Sleeping for 840s...
INFO:pleroma_bot:Sleeping for 840s...

I did several runs and I think the sleeping behavior is working well. 👍
However, I ran into a new issue a couple of times already: The bot appears to be crashing when getting a 504 response from the media endpoint:
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): t.co:443
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FX1qTjBaUAA3aB8.jpg HTTP/1.1" 200 133790
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FXxQ_9-acAAY3Lq.jpg HTTP/1.1" 504 357
Processing tweets... :   0%|                                                            | 0/2485 [06:06<?, ?it/s]
✖ 2023-01-23 05:09:50,492 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:719)

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 427, in _download_media
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 145, in process_tweets
    _download_media(self, media, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 446, in _download_media
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg
ERROR:pleroma_bot:Exception occurred for user, skipping...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 427, in _download_media
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 145, in process_tweets
    _download_media(self, media, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 446, in _download_media
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 655, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: https://pbs.twimg.com/media/FXxQ_9-acAAY3Lq.jpg

Since I will be doing a quite a bit of imports like this, I guess we will be able to clear a few edges cases like this one. Do you think you can take a look? I'll continue running tests with any new version you would provide if it helps you as well.

Interesting, perhaps we can mitigate the 504's when downloading media by reusing the custom session adapter we create for Twitter's API queries (which includes additional retries and handling other error codes as well).
Would you mind giving 1.2.1rc11 a try and see if you notice any improvement?:
 pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc11

Hi @robertoszek,
It looks like I am not getting the 504 anymore with 1.2.1rc11, I'll keep watch to see if this happens again, so that may be improved with this release already! Thank you.
Meanwhile I have been getting another error, which seems to be while Posting Tweets. I am getting hit with a read timeout 404 error from the GET request on the Mastodon media API, while posting tweets.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): myinstanceurl:443 DEBUG:urllib3.connectionpool:https://myinstanceurl:443 "GET /api/v1/media HTTP/1.1" 404 None ✖ 2023-01-24 06:24:10,803 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:722)
The bot crashes here with the following traceback:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.10/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.10/ssl.py", line 1130, in read
    return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='myinstanceurl', port=443): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 689, in main
    post_id = user.post(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 792, in post
    post_id = self.post_pleroma(tweet, poll, sensitive, media, cw=cw)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_pleroma.py", line 267, in post_pleroma
    response = pleroma_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_pleroma.py", line 40, in pleroma_api_request
    response = session.request(
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='myinstanceurl', port=443): Read timed out. (read timeout=30)
ERROR:pleroma_bot:Exception occurred for user, skipping...
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.10/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.10/ssl.py", line 1130, in read
    return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='myinstanceurl', port=443): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 689, in main
    post_id = user.post(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 792, in post
    post_id = self.post_pleroma(tweet, poll, sensitive, media, cw=cw)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_pleroma.py", line 267, in post_pleroma
    response = pleroma_api_request(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_pleroma.py", line 40, in pleroma_api_request
    response = session.request(
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='myinstanceurl', port=443): Read timed out. (read timeout=30)

Any ideas you might have?

Following up on my comment above, I think there's something up with this version. The 504 doesn't occur anymore but it takes forever to process the Tweets (although its probably expected with 3200 tweets ([3200/300]=10 batches with 15 min wait in between, 150min of dead time T_T).
The [00:00<?, ?it/s] counter isn't working/moving like on the stable release, the sleeping info message is duplicated/tripled) - and when it's time to post the tweets I'm getting the read error mentioned above. It hurts lol.
In my use case I expect to bring quite a few users with 3200 tweets, I don't mind the wait time but if it crashes it might be tough.
2022-01-01
⚠ 2023-01-24 07:19:16,167 - pleroma_bot - WARNING - Raising max_tweets to the maximum allowed value (_utils.py:615)
Gathering tweets... 3209
ℹ 2023-01-24 07:20:01,858 - pleroma_bot - INFO - tweets gathered: 	 3209
Processing tweets... :   0%|                                                                                | 0/3209 [00:00<?, ?it/s]ℹ 2023-01-24 07:21:15,346 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:35:02 UTC
ℹ 2023-01-24 07:21:15,347 - pleroma_bot - INFO - Sleeping for 829s...
ℹ 2023-01-24 07:21:15,656 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:35:02 UTC
ℹ 2023-01-24 07:21:15,656 - pleroma_bot - INFO - Sleeping for 828s...
ℹ 2023-01-24 07:21:15,750 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:35:02 UTC
ℹ 2023-01-24 07:21:15,750 - pleroma_bot - INFO - Sleeping for 828s...
ℹ 2023-01-24 07:21:28,082 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:35:02 UTC
ℹ 2023-01-24 07:21:28,082 - pleroma_bot - INFO - Sleeping for 816s...
ℹ 2023-01-24 07:36:03,626 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:50:04 UTC
ℹ 2023-01-24 07:36:03,627 - pleroma_bot - INFO - Sleeping for 842s...
ℹ 2023-01-24 07:36:04,035 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:50:04 UTC
ℹ 2023-01-24 07:36:04,036 - pleroma_bot - INFO - Sleeping for 842s...
ℹ 2023-01-24 07:36:04,693 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:50:04 UTC
ℹ 2023-01-24 07:36:04,693 - pleroma_bot - INFO - Sleeping for 841s...
ℹ 2023-01-24 07:36:04,737 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 07:50:04 UTC
ℹ 2023-01-24 07:36:04,737 - pleroma_bot - INFO - Sleeping for 841s...
ℹ 2023-01-24 07:51:00,880 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:05:06 UTC
ℹ 2023-01-24 07:51:00,880 - pleroma_bot - INFO - Sleeping for 847s...
ℹ 2023-01-24 07:51:01,172 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:05:06 UTC
ℹ 2023-01-24 07:51:01,172 - pleroma_bot - INFO - Sleeping for 847s...
ℹ 2023-01-24 07:51:01,290 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:05:06 UTC
ℹ 2023-01-24 07:51:01,290 - pleroma_bot - INFO - Sleeping for 847s...
ℹ 2023-01-24 07:51:04,408 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:05:06 UTC
ℹ 2023-01-24 07:51:04,409 - pleroma_bot - INFO - Sleeping for 844s...
ℹ 2023-01-24 08:06:06,751 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:20:08 UTC
ℹ 2023-01-24 08:06:06,751 - pleroma_bot - INFO - Sleeping for 843s...
ℹ 2023-01-24 08:06:06,896 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:20:08 UTC
ℹ 2023-01-24 08:06:06,897 - pleroma_bot - INFO - Sleeping for 843s...
ℹ 2023-01-24 08:06:07,341 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:20:08 UTC
ℹ 2023-01-24 08:06:07,341 - pleroma_bot - INFO - Sleeping for 843s...
ℹ 2023-01-24 08:06:08,136 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:20:08 UTC
ℹ 2023-01-24 08:06:08,136 - pleroma_bot - INFO - Sleeping for 842s...
ℹ 2023-01-24 08:20:54,980 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:35:10 UTC
ℹ 2023-01-24 08:20:54,980 - pleroma_bot - INFO - Sleeping for 857s...
ℹ 2023-01-24 08:20:55,049 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:35:10 UTC
ℹ 2023-01-24 08:20:55,049 - pleroma_bot - INFO - Sleeping for 857s...
ℹ 2023-01-24 08:20:55,431 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:35:10 UTC
ℹ 2023-01-24 08:20:55,431 - pleroma_bot - INFO - Sleeping for 857s...
ℹ 2023-01-24 08:20:58,658 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:35:10 UTC
ℹ 2023-01-24 08:20:58,658 - pleroma_bot - INFO - Sleeping for 853s...
ℹ 2023-01-24 08:36:12,944 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:50:12 UTC
ℹ 2023-01-24 08:36:12,944 - pleroma_bot - INFO - Sleeping for 841s...
ℹ 2023-01-24 08:36:13,033 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:50:12 UTC
ℹ 2023-01-24 08:36:13,033 - pleroma_bot - INFO - Sleeping for 841s...
ℹ 2023-01-24 08:36:17,755 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:50:12 UTC
ℹ 2023-01-24 08:36:17,755 - pleroma_bot - INFO - Sleeping for 836s...
ℹ 2023-01-24 08:36:20,916 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 08:50:12 UTC
ℹ 2023-01-24 08:36:20,916 - pleroma_bot - INFO - Sleeping for 833s...
ℹ 2023-01-24 08:51:07,559 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:05:14 UTC
ℹ 2023-01-24 08:51:07,560 - pleroma_bot - INFO - Sleeping for 848s...
ℹ 2023-01-24 08:51:07,726 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:05:14 UTC
ℹ 2023-01-24 08:51:07,727 - pleroma_bot - INFO - Sleeping for 848s...
ℹ 2023-01-24 08:51:08,119 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:05:14 UTC
ℹ 2023-01-24 08:51:08,119 - pleroma_bot - INFO - Sleeping for 848s...
ℹ 2023-01-24 08:51:12,048 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:05:14 UTC
ℹ 2023-01-24 08:51:12,048 - pleroma_bot - INFO - Sleeping for 844s...
Processing tweets... :  25%|█████████████████▌                                                    | 803/3209 [1:45:50<5:17:08,  7.91s/it]ℹ 2023-01-24 09:06:06,018 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:20:16 UTC
ℹ 2023-01-24 09:06:06,019 - pleroma_bot - INFO - Sleeping for 852s...
ℹ 2023-01-24 09:06:06,214 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:20:16 UTC
ℹ 2023-01-24 09:06:06,214 - pleroma_bot - INFO - Sleeping for 852s...
ℹ 2023-01-24 09:06:06,622 - pleroma_bot - INFO - Rate limit exceeded. 0 out of 300 requests remaining until 2023-01-24 09:20:16 UTC
ℹ 2023-01-24 09:06:06,622 - pleroma_bot - INFO - Sleeping for 851s...
Processing tweets... : 100%|███████████████████████████████████████████████████████████████████████| 3209/3209 [2:01:27<00:00,  2.27s/it]
ℹ 2023-01-24 09:21:29,642 - pleroma_bot - INFO - tweets to post: 	 2411
Posting tweets... :   0%|                                                                                | 0/2411 [00:00<?, ?it/s]✖ 2023-01-24 09:22:00,131 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:722)

Regarding the multiple lines of "Rate limit exceeded"/"Sleeping":
By default the bot splits the work among as many logical threads as half of your processor cores.

cores / 2 if cores > 4 else 4
If you have more than 4 cores, take half. If not, split it into 4 threads by default.

That way the bot can work in parallel making requests and processing tweets (regex matching, substituting, etc) and take only a portion of the time it would have taken if it did it single-threaded instead.
If you hit a rate limit and have, let's say 4 parallel threads making requests, they all will fail at roughly the same time and have to wait until the reset time.
Hence the multiple messages at once (I may try to make this more friendly/elegant, I haven't figured out yet exactly how).
Ah, I think I see what may have happened here:
DEBUG:urllib3.connectionpool:https://myinstanceurl:443 "GET /api/v1/media HTTP/1.1" 404 None
A rogue debug GET statement wasn't commented out, most likely.
Any better luck with 1.2.1rc17?:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc17

Hi @robertoszek I hope you are doing well.
I took a break after learning about the API changes brought by Twitter, however I am noticing that the bot seems to be working fine, in most case with a few hundreds of imports. However I am still running into some crashes with bigger batches, I have not encountered the 404 issue mentioned above since then, but I will let you know if I do.
Here's the log for my most recent crash, I would greatly appreciate if you could look into it. This is on a batch of 2000 tweets, during the Processing step, at about 50% progress:
Context:
DEBUG:pleroma_bot:1637850100155420673
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FrrP-3XaEAMQayU.jpg HTTP/1.1" 200 201479
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1640223930597400576?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 200 615
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FsM--CMaUAA1dpe.jpg HTTP/1.1" 200 207210
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FrrP-3XaQAAiJFy.jpg HTTP/1.1" 200 209466
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FsM--CIaQAAoWd5.jpg HTTP/1.1" 200 184088
DEBUG:pleroma_bot:1640733812996071424
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com:443
DEBUG:urllib3.connectionpool:https://pbs.twimg.com:443 "GET /media/FrrP-3WaYAIT6bp.jpg HTTP/1.1" 200 180186
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pbs.twimg.com:443
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "GET /2/tweets/1640730666861223937?poll.fields=duration_minutes%2Cend_datetime%2Cid%2Coptions%2Cvoting_status&media.fields=duration_ms%2Cheight%2Cmedia_key%2Cpreview_image_url%2Ctype%2Curl%2Cwidth%2Cpublic_metrics%2Calt_text&expansions=attachments.poll_ids%2Cattachments.media_keys%2Cauthor_id%2Centities.mentions.username%2Cgeo.place_id%2Cin_reply_to_user_id%2Creferenced_tweets.id%2Creferenced_tweets.id.author_id&tweet.fields=attachments%2Cauthor_id%2Ccontext_annotations%2Cconversation_id%2Ccreated_at%2Centities%2Cgeo%2Cid%2Cin_reply_to_user_id%2Clang%2Cpublic_metrics%2Cpossibly_sensitive%2Creferenced_tweets%2Csource%2Ctext%2Cwithheld HTTP/1.1" 200 203
Processing tweets... :  50%|██████████████████████████                          | 1164/2327 [2:16:14<2:16:07,  7.02s/it]
✖ 2023-03-29 03:06:47,570 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:721)

Error log
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 298, in _get_rt_text
    text = f"{prefix} {tweet_ref['data']['text']}"
KeyError: 'data'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 657, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
KeyError: 'data'
ERROR:pleroma_bot:Exception occurred for user, skipping...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 110, in process_tweets
    tweet["text"] = _get_rt_text(self, tweet)
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 298, in _get_rt_text
    text = f"{prefix} {tweet_ref['data']['text']}"
KeyError: 'data'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 657, in main
    tweets_to_post = process_parallel(
  File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
    for idx, res in enumerate(
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
KeyError: 'data'

Thank you in advance for your help.

Hi @robertoszek, hope you are doing well. Hope you can follow up whenever you get a chance!


RKE Azure - allocated resources left in Azure after deleting the Azure cluster

What kind of request is this (question/bug/enhancement/feature request):
Bug in Rancher: master-head bc3700f8
Steps to reproduce (least amount of steps as possible):
Run the same steps in both v2.2.7 and Rancher: master-head bc3700f8 :

add an Azure node template, in the template, set a new resource group such that Azure will create a dedicated  resource group for this cluster
add a single-node RKE Azure cluster
wait for the cluster be active
check Azure console to see resources are allocated for the cluster under the resource group
delete the cluster in Rancher
check Azure console again

Result:
In v2.2.7, all resources are deleted except the resource group itself

after the cluster is deleted:

in master-head, only the VM is deleted

after the cluster is deleted:

@cjellick this is likely related to my changes docker machine for managed disks, more specifically the azure library updates.

I think this is pre-existing issue. It's also in v2.2.7 did we backport that too @rmweir ?

No we did not, but I believe @jiaqiluo tried to reproduce with 2.2.7?

My bad @jiaqiluo v2.2.7 screenshot still shows all the objects so I was confused.

The bug fix is validated in Rancher:master-head 19e84cee4
Steps:

add an Azure node template, in the template, set a new resource group such that Azure will create a dedicated resource group for this cluster, set disk type to "Managed Disks".
add a single-node RKE Azure cluster
wait for the cluster be active
check Azure console to see resources are allocated for the cluster under the resource group
delete the cluster in Rancher
check Azure console again

Results:
All resource except the managed disk is deleted.
The disk issue is tracked here https://github.com/rancher/rancher/issues/22345


[bandcamp] wrong title for albums, programming help

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.08.13. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.08.13

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[x] Question
[ ] Other

Bandcamp titles are not extracted in albums, e.g. http://music.businesscasual.biz/album/--8 only extracts the URLs. I want to extract the title but am not sure how to do it.
On this line:
https://github.com/rg3/youtube-dl/blob/485047854376465f95309daad4966971f56728ef/youtube_dl/extractor/bandcamp.py#L245
the div with the title is extracted already. I thought about using _html_search_meta, but it seems it only searches for the content attribute and not the actual innerText of the element – I'd like to extract it from the itemprop="name" selector.
bandcamp html:

Is there another way to do this or do I need to extend the beautiful RegExp (still missing a proper HTML parser instead of using RegExp for this...)?

_search_regex.


Disable INFO logging in abseil for emscripten builds

Motivation

Remove the abseil output from sorbet.run.
Test plan

See included automated tests.

Nice. Did you test this locally? You should be able to copy the wasm payload into a local checkout of sorbet.run

Yes, I built the artifact on linux and checked it with my local checkout of sorbet.run 👍


Add a new contact feature

As a busy university student, I can easily add new contacts so that I can keep track of details from peers I have crossed paths with.

This issue no longer fits the project as the idea of project changed. Closed for ease of distraction.

This issue no longer fits the project as the idea of project changed. Closed for ease of distraction.


Force redirect to setup page if no settings are there

Some people seem to upload the CMS without any setup so that the setup page is open to the public. With a forced redirect you cannot access the content-pages without finishing the setup.

deployed with version 1.3.4


Using fragments prevents using cache

I'm trying to create a sample app for others that want to use graphql-flutter and artemis, while learning how to use them myself.
It appears that fragments can only be used with FetchPolicy.noCache (not even FetchPolicy.networkOnly. Removing the cache eliminates a huge part of what makes graphl-flutter appealing, so is there any way to use artemis with fragments and cache?
In main.dart
  await initHiveForFlutter();

In app.dart
return GraphQLProvider(
      client: ValueNotifier(GraphQLClient(
        cache: GraphQLCache(store: HiveStore()),
        link: HttpLink("https://graphql-pokeapi.graphcdn.app"),
      )),
      child: MaterialApp(
        title: "Pokemon Demo",
        onGenerateRoute: (settings) {
          switch (settings.name) {
            case '/':
            default:
              return MaterialPageRoute(builder: (_) => PokemonListScreen());
          }
        },
      ),
    );

In pokemon_list.query.graphql
query PokemonList {
    pokemons(limit: 50) {
        count
        results {
            id
            ...pokemonListCard_item
        }
    }
}

In pokemon_list_card.fragment.graphql
fragment pokemonListCard_item on PokemonItem {
    url
    name
    image
    artwork
}

In pokemon_list.dart
return Query(
        options: QueryOptions(
          document: POKEMON_LIST_QUERY_DOCUMENT,
          operationName: POKEMON_LIST_QUERY_DOCUMENT_OPERATION_NAME,
          fetchPolicy: FetchPolicy.noCache, // This prevents using the cache!!
        ),
        builder: (QueryResult result, {fetchMore, refetch}) {
          if (result.isLoading) return CircularProgressIndicator();
          if (result.hasException) return Center(child: Text(result.exception!.toString()));

          print("result is $result");
          // >>>>>result.data is missing all fragment data with a different fetch policy <<<<<
          final data = PokemonList$Query.fromJson(result.data!);

          final cardList = data.pokemons!.results!.map((pokemon) {
            print("Pokemon name: ${pokemon!.name}");
            return PokemonListCard(itemFrag: pokemon);
          }).toList();

          return RefreshIndicator(
              child: ListView(children: cardList),
              onRefresh: () async {
                await refetch!();
              });
        });

You can find the example project that replicates this problem here https://github.com/baconcheese113/graphql-flutter-artemis-example

https://github.com/comigor/artemis/issues/350

Okay...12 hours straight debugging this problem 🎉
Long Explanation
While breakpoint debugging the internals of  graphql-flutter, normalize, and Hive I noticed that query_manager.dart#L255 attemptCacheWriteFromResponse() fails to write the query to the cache with the fragmentNodes.
The strange part is that mapFetchResultToQueryResult() maps response with the fragment information correctly. But since attemptCacheWriteFromResponse() returns true, it overwrites queryResult a few lines later on line 279
Eventually I dug all the way to normalizes normalizeNode() function used to resolve attemptCacheWriteFromResponse() and noticed that dataForNode is passed in with the correct fragment data, but it is stripped out before the deepMerge() in the same function.
At this point I decided to skim through the normalize README and this caught my attention
The normalize function only normalizes entities that include a __typename field and a valid ID.

So I added the id field to my fragment and everything worked perfectly. I also tried removing the id field and replacing with __typename but that didn't work. I then tried removing id from the results field in the main query and that removed the entire results field from the returned data.
TLDR
Every field that returns a list of objects must include the id field in order to be stored in the cache properly. If stored incorrectly it will mess up the data returned from the network as well.
Prevention
I'm not sure how difficult this would be, but if there was a way to add id to every requested field that has an id the same way __typename is added then nobody would run into this error in the future. I believe this is what Relay does and why users don't necessarily need to add __typename and id everywhere, but I could be mistaken. I imagine there are a lot of bugs in current Artemis projects that haven't been caught due to this issue

Any plans to release a new stable version? I'm not sure if all users are aware there are large differences between the stable and beta version that they need to read a different readme to discover, I wasn't


Spacing between button and title in toolbar

There should be 24px of spacing between icon buttons, and 32px between an icon button and the title.
https://material.io/design/components/app-bars-top.html#specs
Right now, there's no default spacing.

I guess as much as possible, I'd like the framework to take care of instances where I'd otherwise just have to look up the spec and blindly apply it (like I did here). I generally file bugs like this when I've come up with a global override that takes care of the issue and could thus be upstreamed and take care of the issue for more people.
To me, this seems similar to the spacing in a mat-list-item with an avatar, where it was so nice that I could just stick mat-avatar on my img and it's just magically right.
Maybe there could be a mat-toolbar-action class or selector for adding the appropriate spacing?
.mat-toolbar-action:not(:last-child) {
  margin-right: 24px;
}
.mat-toolbar-action + h1 {
  margin-left: 12px;
}

It's totally up to you though, of course. You've got way more knowledge of what has/hasn't worked in the past, and if you think this isn't worth pursuing I'm fine with that.

(revisiting older issues as part of a triaging fix-it)
Going to close this since we're not going to do much more with mat-toolbar going forward since the Material Design spec moved away from it. We might do something different for a separate "top app-bar", but TBD on that front.


refactor(ng-ovh-payment-method): rework payment method implementation

:boom: Breaking changes

removal of ovhPaymentMethodProvider. This provider only provided a way to set target. As the module now use @ovh-ux/manager-core as dependencies target is now available through
@ovh-ux/manager-core region ;
removal of ovhPaymentMethodAvailableTypes. You can now use ovhPaymentMethodRegister component.

:rocket: Feature

create a set of directives usable on any page of the manager for payment method registration.

:recycle: Refactor

use class with methods to use payment methods params easily ;
add enums and define them as angular constants ;
depreciate some methods...

CDS Report build-monorepository#6035.0 ✘
*

Build ✘

CDS Report build-monorepository#6035.1 ✘
*

Build ✘

CDS Report build-monorepository#6036.0 ✘
*

Build ✘

CDS Report build-monorepository#6127.0 ✘
*

Build ✘

CDS Report build-monorepository#6127.2 ✘
*

Build ✘

CDS Report build-monorepository#6142.0 ✘
*

Build ✘

CDS Report build-monorepository#6142.1 ✘
*

Build ✘


Switch DLIO Profiler to DFTracer.

This PR switches DLIO Profiler to DFTracer. Should be merged once the migration is finished.

Profiling was disabled during CI checks -- please see #209


Add a Meridio adapted initContainer image

Description
An initContainer is used to set sysctl's in the load-balancer and proxy PODs. It is currently a BusyBox image, not related to any other Meridio image. This PR adds an init image based on the Meridio base image (same as all other Meridio images).
The initContainer will get more responsibilities in the future, e.g. to setup a tunnel device in test environments that have no secondary networks, like public-cloud and "standard" test clusters. Therefore a meridio-init.sh is included for now, but more advanced programs can be added if necessary.
The first commit in this PR only builds and uses the new init image. The start commands (a bunch of sysctl -w) is not altered. However the meridio-init.sh is prepared to handle start without parameters.
      initContainers:
        - name: init
          image: registry.nordix.org/cloud-native/meridio/init:latest
          securityContext:
            privileged: true

In such case the script setup depending on hostname.
if test -n "$1"; then
	cmd=$1
	shift
else
	cmd=$INIT_FUNCTION
fi
if ! test -n "$cmd"; then
	# Make a guess from the hostname
	if hostname | grep -qE 'load-balance|stateless-lb'; then
		cmd=lb
	elif hostname | grep -q 'proxy'; then
		cmd=proxy
	fi
fi

It is however better to explicitly specify the function in an environment variable.
      initContainers:
        - name: init
          image: registry.nordix.org/cloud-native/meridio/init:latest
          env:
            - name: INIT_FUNCTION
              value: "proxy"
          securityContext:
            privileged: true

Tunnel setup
As mentioned above the first use of the init image is to setup a tunnel (vxlan) to the load-balancer (instead of a real external interface). The configuration may look something like:
      initContainers:
        - name: init
          image: registry.nordix.org/cloud-native/meridio/init:latest
          env:
            - name: INIT_FUNCTION
              value: "lb"
             - name: TUNNEL_PARAMS
              value: "<IP_ADDRESS>,100"
         securityContext:
            privileged: true

The params are the the POD address of the local test-generator (TG) and the vxlan id. The id will be handled in the same way as the vlan-tag for test compatibility reasons. So in this example the tunnel device will be called ext-vlan.100, same as for vlan setup.
Issue link
N/A
Checklist

Purpose

[ ] Bug fix
[x] New functionality
[ ] Documentation
[ ] Refactoring
[ ] CI

Test

[ ] Unit test
[x] E2E Test
[x] Tested manually

Introduce a breaking change

[ ] Yes (description required)
[x] No

Introduce changes in the Operator

[x] Yes (description required)
[ ] No

A better place than hack/meridio-init.sh is requested.

@LionelJouin Will the addition of an init image require CI updates?

Yes, I could update the CI.

The tunnel procedure should be unified with
https://github.com/Nordix/k8s-service-tunnel

The init-container has been updated to be more like https://github.com/Nordix/k8s-service-tunnel.
The $TUNNEL_PARAMS is removed and is replaced by more clear TUNNEL_* variables. The function is tested in xcluster, using ovl/forwarder-test the external connectivity can be selected with --exconnect=vlan|multus|tunnel for the same tests. There were a few suprises along the way, and based on new experiences this is what has to be specified;
      initContainers:
        - name: meridio-init
          image: registry.nordix.org/cloud-native/meridio/init:local
          securityContext:
            privileged: true
          env:
            - name: INIT_FUNCTION
              value: lb
            - name: TUNNEL_PEER
              value: "<IP_ADDRESS>"
            - name: TUNNEL_ID
              value: "101"
            - name: TUNNEL_DEV
              value: "nsm-1"
            - name: TUNNEL_IPV4
              value: "<IP_ADDRESS>/24"
            - name: TUNNEL_IPV6
              value: "fd00::<IP_ADDRESS>/120"

The TUNNEL_ID (VNI) and the addresses must be coordinated with the external machine that has the "other" tunnel endpoint. This means that an IPAM in the cluster will not help. The addresses can be about anything that not causes problems on the external machine.
The VNI must be specified for the same reasons. If the external machine has several tunnels (representing secondary networks) they must have different VNIs.
The TUNNEL_DEV is what the FE sees and must be the same as the NFE_EXTERNAL_INTERFACE.

The operator config may be extended with something like;
  interface:
    name: tunnel0
    type: tunnel
    tunnel:
      peer: "<IP_ADDRESS>"
      id: "101"
      local-ipv4: "<IP_ADDRESS>/24"
      local-ipv6: "fd00::<IP_ADDRESS>/120"

Testing revealed that sometimes the external tunnel end must be "excercised" to trig some UDP traffic so the connection through the service is setup. A ping towards the TUNNEL_IPV4 on the cluster-end of the tunnel is sufficient.


feature: allow calling [Command]s on server-only directly. #3450

TODO

[] IL2CPP builds still fail
[ ] test coverage
[ ] try in assets: ummorpg and replace the old wrappers
[ ] change to isServer / is Client?

before:

    [Command]
    private void CmdTest(int value)
    {
        NetworkWriterPooled writer = NetworkWriterPool.Get();
        writer.WriteInt(value);
        SendCommandInternal("System.Void Mirror.Examples.Tanks.Tank::CmdTest(System.Int32)", 897081557, writer, 0);
        NetworkWriterPool.Return(writer);
    }

    public override void OnStartServer()
    {
        CmdTest(42);
    }

after:
    [Command]
    private void CmdTest(int value)
    {
        if (NetworkServer.active && !NetworkClient.active)
        {
            UserCode_CmdTest__Int32(value);
            return;
        }
        NetworkWriterPooled writer = NetworkWriterPool.Get();
        writer.WriteInt(value);
        SendCommandInternal("System.Void Mirror.Examples.Tanks.Tank::CmdTest(System.Int32)", 897081557, writer, 0);
        NetworkWriterPool.Return(writer);
    }

    public override void OnStartServer()
    {
        CmdTest(42);
    }

CmdOnAnimationServerMessage still fails to weave properly:
[Command]
private void CmdOnAnimationServerMessage(int stateHash, float normalizedTime, int layerId, float weight, byte[] parameters)
{
    //IL_001b: Expected O, but got I4
    //IL_001b: Expected F4, but got O
    if (NetworkServer.active && !NetworkClient.active)
    {
        ((NetworkAnimator)/*Error near IL_001b: Stack underflow*/).UserCode_CmdOnAnimationServerMessage__Int32__Single__Int32__Single__Byte[]((int)/*Error near IL_001b: Stack underflow*/, (float)/*Error near IL_001b: Stack underflow*/, (int)/*Error near IL_001b: Stack underflow*/, (float)this, (byte[])stateHash);
        return;
    }
    NetworkWriterPooled writer = NetworkWriterPool.Get();
    writer.WriteInt(stateHash);
    writer.WriteFloat(normalizedTime);
    writer.WriteInt(layerId);
    writer.WriteFloat(weight);
    writer.WriteBytesAndSize(parameters);
    SendCommandInternal("System.Void Mirror.NetworkAnimator::CmdOnAnimationServerMessage(System.Int32,System.Single,System.Int32,System.Single,System.Byte[])", -1895293543, writer, 0);
    NetworkWriterPool.Return(writer);
}


Last step - unclear

Hello there! :)
Firstly, thanks for the tool, it's looking promising! I was about to test it, and got it all except the last step. What should I do exactly in this step:
Add Post-Receive Webhook
Finally, add the appropriate one of these URLs as a webhook into your repository settings:

For GitHub: http://slimjim.yourcompany.com/gh_hook

For BitBucket: http://slimjim.yourcompany.com/bb_hook

?
Where should I add the https://mysite/gh_hook?
Thanks in further!

@milosb793 You'd add that in the repo's webhook settings. So for GitHub it would be here:

Hope that helps!

Yeah, that's it! Thank you very much!


fix send from empty wallet

Closes #530
@tiero please review
Before:
https://github.com/vulpemventures/marina/assets/9318412/1b759551-e4b6-4934-9cf5-2bdc9ff3ee63
After:
https://github.com/vulpemventures/marina/assets/9318412/8400ee1a-aa27-4a21-87b7-d0b0fe0766e0

can we avoid the dot after the zero? 0.