German

The metadata is defaulting to german, translating the titles of my documents to German, and also adding tags in german.  Whats going on?

Here is the startup log. Gives no indication until it starts getting into the document catorization
`2024-12-18T14:34:42.060069037Z 2024-12-18T14:34:41: PM2 log: Launching in no daemon mode
2024-12-18T14:34:42.095497912Z 2024-12-18T14:34:42: PM2 log: App [paperless-assistant:0] starting in -cluster mode-
2024-12-18T14:34:42.115070356Z 2024-12-18T14:34:42: PM2 log: App [paperless-assistant:0] online
2024-12-18T14:34:42.358581852Z Server running on port 3000
2024-12-18T14:34:42.362731452Z Setup not completed. Skipping initial scan. Visit http://your-domain-or-ip.com:3000/setup to complete setup.
2024-12-18T14:36:03.937462424Z 2024-12-18T14:36:03: PM2 log: App name:paperless-assistant id:0 disconnected
2024-12-18T14:36:03.939553756Z 2024-12-18T14:36:03: PM2 log: App [paperless-assistant:0] exited with code [0] via signal [SIGINT]
2024-12-18T14:36:03.941253248Z 2024-12-18T14:36:03: PM2 log: App [paperless-assistant:0] will restart in 100ms
2024-12-18T14:36:04.043498721Z 2024-12-18T14:36:04: PM2 log: App [paperless-assistant:0] starting in -cluster mode-
2024-12-18T14:36:04.085356084Z 2024-12-18T14:36:04: PM2 log: App [paperless-assistant:0] online
2024-12-18T14:36:04.280250044Z Server running on port 3000
2024-12-18T14:36:05.875115246Z Running initial scan...
2024-12-18T14:36:05.875150972Z Starting document scan...
2024-12-18T14:36:06.916171787Z Found 0 existing tags
2024-12-18T14:36:07.736600970Z Fetched page 1, got 100 documents. Total so far: 100
2024-12-18T14:36:08.165297458Z Fetched page 2, got 25 documents. Total so far: 125
2024-12-18T14:36:08.165602449Z Processing new document: NIST.SP.1299
2024-12-18T14:36:09.890648804Z Refreshing tag cache...
2024-12-18T14:36:09.923839649Z Tag cache refreshed. Found 0 tags.
2024-12-18T14:36:10.264242776Z Successfully created tag "Cybersecurity" with ID 1
2024-12-18T14:36:10.407278296Z Successfully created tag "Framework" with ID 2
2024-12-18T14:36:10.572692275Z Successfully created tag "NIST" with ID 3
2024-12-18T14:36:10.739694911Z Successfully created tag "Risk Management" with ID 4
2024-12-18T14:36:10.893722268Z Created new correspondent "National Institute of Standards and Technology (NIST)" with ID 1
2024-12-18T14:36:10.953600834Z Current tags for document 86: []
2024-12-18T14:36:10.954257068Z Adding new tags: [ 1, 2, 3, 4 ]
2024-12-18T14:36:10.954584156Z Combined tags: [ 1, 2, 3, 4 ]
2024-12-18T14:36:11.438857601Z Updated document 86 with: {
2024-12-18T14:36:11.438884590Z   tags: [ 1, 2, 3, 4 ],
2024-12-18T14:36:11.438888289Z   title: 'NIST Cybersecurity Framework 2.0: Resource & Overview Guide',
2024-12-18T14:36:11.438891498Z   created: '2024-02-01T00:00:00.000Z',
2024-12-18T14:36:11.438894256Z   correspondent: 1,
2024-12-18T14:36:11.438896885Z   language: 'de'
2024-12-18T14:36:11.438899487Z }
2024-12-18T14:36:11.473073102Z Updated document NIST.SP.1299 with: {
2024-12-18T14:36:11.473107697Z   tags: [ 1, 2, 3, 4 ],
2024-12-18T14:36:11.473111756Z   title: 'NIST Cybersecurity Framework 2.0: Resource & Overview Guide',
2024-12-18T14:36:11.473114522Z   created: '2024-02-01',
2024-12-18T14:36:11.473116626Z   correspondent: 1,
2024-12-18T14:36:11.473118724Z   language: 'de'
2024-12-18T14:36:11.473120782Z }
2024-12-18T14:36:11.473715537Z Document 86 added to processed_documents
2024-12-18T14:36:11.473840775Z Document NIST Cybersecurity Framework 2.0: Resource & Overview Guide processing completed
2024-12-18T14:36:11.473938282Z Processing new document: JOINT-CSI-BEST-PRACTICES-EVENT-LOGGING-THREAT-DETECTION
2024-12-18T14:36:13.653211817Z Found tag "Cybersecurity" in cache with ID 1
2024-12-18T14:36:13.802378032Z Successfully created tag "Event Logging" with ID 5
2024-12-18T14:36:13.946775416Z Successfully created tag "Threat Detection" with ID 6
2024-12-18T14:36:14.100959674Z Successfully created tag "Best Practices" with ID 7
2024-12-18T14:36:14.266927424Z Created new correspondent "Australian Government" with ID 2
2024-12-18T14:36:14.319837552Z Current tags for document 36: []
2024-12-18T14:36:14.319861067Z Adding new tags: [ 1, 5, 6, 7 ]
2024-12-18T14:36:14.319967293Z Combined tags: [ 1, 5, 6, 7 ]
2024-12-18T14:36:14.679988314Z Updated document 36 with: {
2024-12-18T14:36:14.680017705Z   tags: [ 1, 5, 6, 7 ],
2024-12-18T14:36:14.680023966Z   title: 'Best Practices für Ereignisprotokollierung und Bedrohungserkennung',
2024-12-18T14:36:14.680029102Z   created: '2024-01-01T00:00:00.000Z',
2024-12-18T14:36:14.680033317Z   correspondent: 2,
2024-12-18T14:36:14.680037327Z   language: 'de'
2024-12-18T14:36:14.680041356Z }
2024-12-18T14:36:14.727501140Z Updated document JOINT-CSI-BEST-PRACTICES-EVENT-LOGGING-THREAT-DETECTION with: {
2024-12-18T14:36:14.727536252Z   tags: [ 1, 5, 6, 7 ],
2024-12-18T14:36:14.727545692Z   title: 'Best Practices für Ereignisprotokollierung und Bedrohungserkennung',
2024-12-18T14:36:14.727556552Z   created: '2024-01-01',
2024-12-18T14:36:14.727563181Z   correspondent: 2,
2024-12-18T14:36:14.727569563Z   language: 'de'
2024-12-18T14:36:14.727575923Z }
2024-12-18T14:36:14.727701184Z Document 36 added to processed_documents
2024-12-18T14:36:14.727814205Z Document Best Practices für Ereignisprotokollierung und Bedrohungserkennung processing completed
2024-12-18T14:36:14.728007099Z Processing new document: AIDesignLLMs786734
2024-12-18T14:36:16.599887946Z Successfully created tag "Künstliche Intelligenz" with ID 8
2024-12-18T14:36:16.755588599Z Successfully created tag "Sprachmodelle" with ID 9
2024-12-18T14:36:16.929326580Z Successfully created tag "Designmuster" with ID 10
2024-12-18T14:36:17.053373478Z Successfully created tag "Generative KI" with ID 11
2024-12-18T14:36:17.198389846Z Created new correspondent "Gartner, Inc." with ID 3
2024-12-18T14:36:17.260567859Z Current tags for document 31: []
2024-12-18T14:36:17.260731878Z Adding new tags: [ 8, 9, 10, 11 ]
2024-12-18T14:36:17.260984969Z Combined tags: [ 8, 9, 10, 11 ]
2024-12-18T14:36:18.577392246Z Updated document 31 with: {
2024-12-18T14:36:18.577421332Z   tags: [ 8, 9, 10, 11 ],
2024-12-18T14:36:18.577428253Z   title: 'AI Design Patterns für große Sprachmodelle',
2024-12-18T14:36:18.577433877Z   created: '2023-01-01T00:00:00.000Z',
2024-12-18T14:36:18.577438548Z   correspondent: 3,
2024-12-18T14:36:18.577443069Z   language: 'de'
2024-12-18T14:36:18.577449279Z }
2024-12-18T14:36:18.613886846Z Updated document AIDesignLLMs786734 with: {
2024-12-18T14:36:18.613903444Z   tags: [ 8, 9, 10, 11 ],
2024-12-18T14:36:18.613906390Z   title: 'AI Design Patterns für große Sprachmodelle',
2024-12-18T14:36:18.613909262Z   created: '2023-01-01',
2024-12-18T14:36:18.613911947Z   correspondent: 3,
2024-12-18T14:36:18.613914146Z   language: 'de'
2024-12-18T14:36:18.613916294Z }
`

Thanks for reporting this issue. I have an idea where the issue is.
I will get back to you in a couple of minutes.

Can you delete the cointainer and also remove all tags and correspondents? Then run a new test and pull the new docker image.
It is probally an issue with the system prompt.

Can you delete the cointainer and also remove all tags and correspondents? Then run a new test and pull the new docker image. It is probally an issue with the system prompt.

Alot of the document titles have also been modified.  Is that going to throw off the prompt?

As soon as you delete the old docker image and pull the new one, the database is empty and it will reiterate through the files.

Maybe I will implement a language setting to specifically tell the prompt what language is needed.
As of now the new system prompt that will be appended to your own custom promp looks like this:

        Return the result EXCLUSIVELY as a JSON object. In addition, fill the data according to the language of the original document.:

        {
          "title": "Title",
          "correspondent": "Correspondent/Author",
          "tags": ["Tag1", "Tag2", "Tag3", "Tag4"],
          "document_date": "YYYY-MM-DD",
          "language": "en/de/es/..."
        }`;

So it tries to find the language and adapt that to the Tags and Title. With the language setting in the future the prompt will be filled with the user desired language.

Can you delete the cointainer and also remove all tags and correspondents? Then run a new test and pull the new docker image. It is probally an issue with the system prompt.

2024-12-18T16:03:22: PM2 log: Launching in no daemon mode
2024-12-18T16:03:22: PM2 log: App [paperless-assistant:0] starting in -cluster mode-
2024-12-18T16:03:22: PM2 log: App [paperless-assistant:0] online
Server running on port 3000
Setup not completed. Skipping initial scan. Visit http://your-domain-or-ip.com:3000/setup to complete setup.
2024-12-18T16:04:37: PM2 log: App name:paperless-assistant id:0 disconnected
2024-12-18T16:04:37: PM2 log: App [paperless-assistant:0] exited with code [0] via signal [SIGINT]
2024-12-18T16:04:37: PM2 log: App [paperless-assistant:0] will restart in 100ms
2024-12-18T16:04:37: PM2 log: App [paperless-assistant:0] starting in -cluster mode-
2024-12-18T16:04:37: PM2 log: App [paperless-assistant:0] online
Server running on port 3000
Running initial scan...
Starting document scan...
Error during document scan: TypeError: Cannot read properties of undefined (reading 'length')
    at scanDocuments (/app/server.js:48:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-12-18T16:05:03: PM2 log: App name:paperless-assistant id:0 disconnected
2024-12-18T16:05:03: PM2 log: App [paperless-assistant:0] exited with code [0] via signal [SIGINT]
2024-12-18T16:05:03: PM2 log: App [paperless-assistant:0] will restart in 150ms
2024-12-18T16:05:03: PM2 log: App [paperless-assistant:0] starting in -cluster mode-
2024-12-18T16:05:03: PM2 log: App [paperless-assistant:0] online
Server running on port 3000
Running initial scan...
Starting document scan...
Error during document scan: TypeError: Cannot read properties of undefined (reading 'length')
    at scanDocuments (/app/server.js:48:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Error fetching documents page 2: Cannot read properties of undefined (reading 'length')
Error fetching documents: TypeError: Cannot read properties of undefined (reading 'length')
    at PaperlessService.getAllDocuments (/app/services/paperlessService.js:188:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/server.js:122:23
Error fetching documents page 2: Cannot read properties of undefined (reading 'length')
Error fetching documents: TypeError: Cannot read properties of undefined (reading 'length')
    at PaperlessService.getAllDocuments (/app/services/paperlessService.js:188:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/server.js:122:23
Error fetching documents page 2: Cannot read properties of undefined (reading 'length')
Error fetching documents: TypeError: Cannot read properties of undefined (reading 'length')
    at PaperlessService.getAllDocuments (/app/services/paperlessService.js:188:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/server.js:122:23
2024-12-18T16:05:52: PM2 log: [PM2][WORKER] Reset the restart delay, as app paperless-assistant has been up for more than 30000ms
Error fetching documents page 2: Cannot read properties of undefined (reading 'length')
Error fetching documents: TypeError: Cannot read properties of undefined (reading 'length')
    at PaperlessService.getAllDocuments (/app/services/paperlessService.js:188:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/server.js:122:23

The error indicates that the file is to big and the token length is exceeded.
I defined gpt-4o-mini as desired model.
Can you give me the prompt you defined in the setup process?

I don't have a prompt.  Just yours.

The error indicates that the file is to big and the token length is exceeded. I defined gpt-4o-mini as desired model.
Can you give me the prompt you defined in the setup process?

I don't have a prompt. Just yours.

The error indicates that the file is to big and the token length is exceeded. I defined gpt-4o-mini as desired model.
Can you give me the prompt you defined in the setup process?

Shoot...I have a ' in there by accident.

The error indicates that the file is to big and the token length is exceeded. I defined gpt-4o-mini as desired model.
Can you give me the prompt you defined in the setup process?

Can you make that a config setting?

The error indicates that the file is to big and the token length is exceeded. I defined gpt-4o-mini as desired model.
Can you give me the prompt you defined in the setup process?

Can you make that a config setting?

Sure, I will update the repo in the next few hours.

How frequently do you poll for new documents being added?

I think it would also be good to have a UI language element as well...

      <% documents.forEach(function(doc) { %>

<%= doc.title %>

Erstellt am: <%= new Date(doc.created).toLocaleDateString('de-DE') %>

Tags:
            <% if (doc.tags && doc.tags.length > 0) { %>
              <% doc.tags.forEach(function(tag) { %>
                <%= tag %>
              <% }); %>
            <% } else { %>
              Keine Tags
            <% } %>

Korrespondent: <%= doc.correspondent || 'Nicht zugewiesen' %>

      <% }); %>
    <% } else { %>

Keine Dokumente gefunden.

    <% } %>

@schergr I implemented some features like:

Prefill Prompt (example prompt that can be altered) with the improvement of using the language of the document for tags and title.
The user can choose a desired OpenAI Model (GPT-3.5, GPT-4o, GPT-4o-mini)

The docker container is also updated. Maybe you can give it a try?

Sure. Happy to.

@schergr I implemented some features like:

Prefill Prompt (example prompt that can be altered) with the improvement of using the language of the document for tags and title.
The user can choose a desired OpenAI Model (GPT-3.5, GPT-4o, GPT-4o-mini)

The docker container is also updated. Maybe you can give it a try?

Sure. Happy to.

Question...if I don't delete the tags and correspondents already in paperless before I download the new container, will it cause issues. In other words, do you store the tags that you create somewhere else, or do you query the paperless tag list for matches so you don't create dupes.

Good question! The processed documents will be stored inside a SQLite3 DB within the container. Only the ID of the processed document is stored.
For example if you run paperless-ai for the first time it will tag all documents and store the id as done. When you delete the container and re run as fresh installation of paperless-ai then the exisiting tags in papeless-ngx will not be touched. Instead it will add new tags and change only the title. So the old tags remain existent. Because the memory of what is already processed is only stored inside the container itself.

Good question! The processed documents will be stored inside a SQLite3 DB within the container. Only the ID of the processed document is stored.
For example if you run paperless-ai for the first time it will tag all documents and store the id as done. When you delete the container and re run as fresh installation of paperless-ai then the exisiting tags in papeless-ngx will not be touched. Instead it will add new tags and change only the title and correspondet (if the AI finds a new variant). So the old tags remain existent. Because the memory of what is already processed is only stored inside the container itself.
But it checks on creation of the tag if it already exists to prevent running in issues with the paperless-ngx api.

So if it looks at a new document, and says hey, here's a tag, but that tag already exists in paperless, it won't try to create a new instance of that tag? It will just apply the exisitng one?

Still spitting out German tags and titles. less so, but still doing that. The document that it categorized this way had nothing in German about it at all.

Still spitting out German tags and titles. less so, but still doing that. The document that it categorized this way had nothing in German about it at all.

I downloaded the update, and also used your suggested prompt. Still using German. Can you change the tags in the code to be in enslish as well?

Sorry you have to clarify something for me. Do you mean with German only this text you showed in the snippet you posted here?

Well, both actually.  But I'm wondering if that has anything to do with it.
On Mon, Dec 23, 2024 at 12:22 PM clusterzx @.***> wrote:

Sorry you have to clarify something for me. Do you mean with German only
this text you showed in the snippet you posted here?
—
Reply to this email directly, view it on GitHub
https://github.com/clusterzx/paperless-ai/issues/2#issuecomment-2560060906,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAOORTGZNY7SSRT32WISUT2HBBFLAVCNFSM6AAAAABT24D3JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRQGA3DAOJQGY
.
You are receiving this because you were mentioned.Message ID:
@.***>

There is no way that it is an error caused by the actual code.
If you use docker make sure to delete the container and the image and pull it new.
Also try it with a test environment with a freshly paperless instance. Maybe it has something to do that the documents are flagged now as German?
I tried with a test instance only with English documents. There was nothing German in the result.


Memory Continually Increasing Intermittently

Hello everyone,
My team seems to have stumbled upon a really weird issue with the conductor python sdk where the memory of the worker continues to climb until the worker runs out of resources (in our case our OpenShift cluster OOMKills the pod due to the resource utilization exceeding it's allocated quota). What is odd is the timing of the memory increasing is not consistent. Sometimes it will start as soon as the worker starts while other times it will start 30 to 45 minutes after the application starts. Additionally, once in a while it seems to not have this issue. I have tried tracing through the polling functionality within the SDK and cannot seem to find anything that stands out as the cause of the memory. Wondering if anyone else is experiencing this issue and what is the root cause?

This graph shows the example code below running and being restarts. When you see a long flat line that is when we turned it off and tried looking at code. Considering this workflow does absolutely nothing (execute doesn't even get called) I wouldn't expect to see a bunch of spikes in memory and this should just be flat for the most part.
Steps to reproduce:

Setup a conductor server with a generic workflow (doesn't have to be anything specific)
Deploy a pod with any number of workers greater than 1 (in our case we did this with 12 workers) using the code shown below.
Ensure that the only package installed (besides base packages) is conductor-python.
Monitor the memory

Sample Code:
`
from conductor.client.automator.task_handler import TaskHandler
from conductor.client.configuration.configuration import Configuration
from conductor.client.http.models import Task, TaskResult
from conductor.client.worker.worker_interface import WorkerInterface
configuration = Configuration(
server_api_url=f"http://localhost:3500/", debug=True
)
class SillyWorker(WorkerInterface):
def execute(self, task: Task) -> TaskResult | None:
return None
def get_polling_interval_in_seconds(self) -> float:
    return 1

workers = [
SillyWorker(task_definition_name="silly-worker-1"),
SillyWorker(task_definition_name="silly-worker-2"),
SillyWorker(task_definition_name="silly-worker-3"),
SillyWorker(task_definition_name="silly-worker-4"),
SillyWorker(task_definition_name="silly-worker-5"),
SillyWorker(task_definition_name="silly-worker-6"),
SillyWorker(task_definition_name="silly-worker-7"),
SillyWorker(task_definition_name="silly-worker-8"),
SillyWorker(task_definition_name="silly-worker-9"),
SillyWorker(task_definition_name="silly-worker-10"),
SillyWorker(task_definition_name="silly-worker-11"),
SillyWorker(task_definition_name="silly-worker-12"),
]
def start_workers():
with TaskHandler(workers, configuration) as task_handler:
    task_handler.start_processes()
    task_handler.join_processes()

if name == "main":
start_workers()
`
Environment:

OpenShift Container Platform
Python 3.11.9
Container image: python:3.11 (based on dockerhub python image)
conductor-python==1.1.7 (have gone back as far as 1.1.4 with same issues)

Hey @ryorke1,
Do you have some kind of proxying for that server_api_url? It's supposed to end with /api because orkes-conductor-server image has proxying for /api internally for API queries - do the logs show that the workers are polling? Does the pod have enough CPU for garbage collection?
Have you tried newer or older python images? IIRC there were changes in 3.11 which were supposed to bring more performance at the cost of a bit more memory, and those changes also initially caused a few leaks in some libraries.
I would appreciate it if you could share Dockerfile + compose yaml for the worker with cpu/memory capped so that we could use them to reproduce the issue.
It does not seem to occur on my machine w/ python 3.11.9/conductor-python=1.1.7 .

Hi @c4lm
here is the dockerfile:
FROM python:3.11

COPY worker.py /app
COPY requirements.txt /app

WORKDIR /app

RUN pip install -r requirements.txt && \
    useradd -m worker

USER worker
CMD ["python", "worker.py"]

There is no compose file as we are running this in OpenShift. Once this image is built we push it into our image repository and deploy it using a generic helm deployment manifest. As for the URL, there is no proxy. In this case the test code was running it on localhost specifically on port 3500. I forgot to add the /api portion in the example as I was cleaning up the code to remove stuff to simplify the example.
Here is the manifest we are using for this testing in OpenShift:
kind: Pod
apiVersion: v1
metadata:
  name: conductor-testing
  namespace: default
spec:
  containers:
    - resources:
        limits:
          cpu: 4
          memory: 1Gi
        requests:
          cpu: 100m
          memory: 400Mi
      name: conductor-testing
      imagePullPolicy: IfNotPresent
      image: 'python:3.11'

We have also tried to reproduce this locally in docker and it takes a bit of restarting and leaving it running but eventually it will have this issue. It's not consistent like it is on OpenShift.
Here is a snippet of the logs received from the example script above:
2024-06-13 13:40:57,530 [193] conductor.client.automator.task_handler INFO     TaskHandler initialized
2024-06-13 13:40:57,531 [193] conductor.client.automator.task_handler INFO     Starting worker processes...
2024-06-13 13:40:57,533 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-1 with domain None with polling interval 1
2024-06-13 13:40:57,534 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-2 with domain None with polling interval 1
2024-06-13 13:40:57,536 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-4 with domain None with polling interval 1
2024-06-13 13:40:57,631 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-10 with domain None with polling interval 1
2024-06-13 13:40:57,723 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-3 with domain None with polling interval 1
2024-06-13 13:40:57,723 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-5 with domain None with polling interval 1
2024-06-13 13:40:57,724 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-6 with domain None with polling interval 1
2024-06-13 13:40:57,725 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:57,725 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:57,728 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-1?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:57,728 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-4?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:57,724 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-7 with domain None with polling interval 1
2024-06-13 13:40:57,823 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-1, worker_id: conductor-testing, domain: None
2024-06-13 13:40:57,823 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-9 with domain None with polling interval 1
2024-06-13 13:40:57,824 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-4, worker_id: conductor-testing, domain: None
2024-06-13 13:40:57,825 [193] conductor.client.automator.task_handler INFO     Started 12 TaskRunner process
2024-06-13 13:40:57,826 [193] conductor.client.automator.task_handler INFO     Started all processes
2024-06-13 13:40:57,923 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-11 with domain None with polling interval 1
2024-06-13 13:40:57,923 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-8 with domain None with polling interval 1
2024-06-13 13:40:57,924 [193] conductor.client.automator.task_runner INFO     Polling task silly-worker-12 with domain None with polling interval 1
2024-06-13 13:40:57,925 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:57,928 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-2?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:57,929 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-2, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,124 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,124 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,127 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-3?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,127 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-11?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,128 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-11, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,128 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-3, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,223 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,323 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,325 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,325 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,424 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-7?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,425 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,428 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,524 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-7, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,524 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-9?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,525 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-9, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,531 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-10?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,532 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-10, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,532 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): localhost:3500
2024-06-13 13:40:58,532 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-5?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,532 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-12?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,534 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-8?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,535 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-8, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,532 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-6?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,623 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-5, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,623 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-12, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,624 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-6, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,826 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-1?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,826 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-4?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,827 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-1, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,827 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-4, worker_id: conductor-testing, domain: None
2024-06-13 13:40:58,932 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-2?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:58,933 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-2, worker_id: conductor-testing, domain: None
2024-06-13 13:40:59,131 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-11?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:59,131 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-11, worker_id: conductor-testing, domain: None
2024-06-13 13:40:59,131 urllib3.connectionpool DEBUG    http://localhost:3500 "GET /api/tasks/poll/silly-worker-3?workerid=conductor-testing HTTP/1.1" 200 2
2024-06-13 13:40:59,132 [193] conductor.client.automator.task_runner DEBUG    Polled task: silly-worker-3, worker_id: conductor-testing, domain: None
...

With regards to the CPU utilization, we have given the pod enough resources based on the metrics from the pod such that the CPU limit shouldn't be an issue as shown in this graph. Right now we are allowing the pod to utilize 1/5 of a CPU but the pod never exceeds this limit. However, that being said, I increased the pod to 4 CPUs and ran it for a short period of time and still see the same results.

Before increasing CPU limits

After increasing CPU limits to 4 cpus
With regards to different version of the conductor-python-sdk, we have tried going back to 1.1.4 but still see the same issues. For the Python version itself, I had not yet tried downgrading from 3.11 to 3.10. I just tried that and no change. I also tried using 3.12 and have the same results.

Using python 3.10.14

Using python 3.12.4
Please let me know if there is anything else I can provide to assist!

@ryorke1
having a bit of a replication crisis over here
Using my compose file, memory usage consistently stays under 97Mb according to docker stats if I let it run for a while - that is even lower than when running locally on my machine!
I also did a bunch of runs with memray locally again

I can see the small memory RSS increase, but not sure if it's indicative of anything since memory was not bounded.
At best this memory graph of one of the forked worker processes might serve as weak evidence that there is a small native memory leak when running on my Mac (which does not happen when running in docker), highest peak was 8mb higher than the starting peak, but Python heap was stable, it changed of course, but the peaks were the same, like 380Kb. Highest amount allocated from a single call was 108 KiB (I ran memray with --trace-python-allocators and with -Xdev on python3).
Everything is complicated by CPython having version-specific leak issues and platform-specific leak issues, and then the libraries having them too (e.g. this one, and it's not the only one).
Regarding the libraries we use here - metrics are disabled, so it can't be prometheus-client, connection is http, so it can't be SSL-related leaks.
So that leaves us with the following options:

rest of the SDK
logging
our client library - requests/urllib3
something from the above and our usage of multiprocessing

Usually whenever I encounter leaks if http requests are involved, it's 1 or 3 (regardless of the language involved), so for now we could stick to that assumption. Rest client is per process, session is per rest client and we do not close it because it is reused, so we should not be leaking there...
I will try to do more runs, and if I still can't replicate, I will try to create a reduced example (essentially just multiprocessing + requests calls) for you to test and reach out.
In the meantime, could you please try the following change (probably should not change anything, but what if):
import requests
configuration = Configuration(
    server_api_url=f"http://localhost:3500/api", debug=False
)
session = requests.Session()
session.mount('http://', requests.adapters.HTTPAdapter(pool_connections=1, pool_maxsize=1))
configuration.http_connection = session

Additionally, it would be great if you could try running the script with memray and capture the results.
e.g.
python3 -m memray run --follow-fork leak.py
it will generate 14 .bin files (1 for parent, 1 for logger, 12 for workers).
you can use these files to create reports
e.g.
python3 -m memray flamegraph --leaks memray-leak.py.49244.bin.49260
You could also try running python3 -Xdev -m memray run --follow-fork --trace-python-allocators leak.py but beware, it produces large binary files.
What I was using in my docker-based attempts:
---
version: '3.9'

services:

  worker-1:
    image: "pythonleak"
    build:
      context: ./
      dockerfile: ./Dockerfile
    networks:
      - internal
    deploy:
      resources:
        limits:
          cpus: '4'
          memory: 1G

networks:
  internal:

FROM python:3.12

RUN mkdir app

COPY ./leak.py /app/

WORKDIR /app

RUN pip install conductor-python && \
    useradd -m worker

USER worker
CMD ["python", "leak.py"]

from conductor.client.automator.task_handler import TaskHandler
from conductor.client.configuration.configuration import Configuration
from conductor.client.http.models import Task, TaskResult
from conductor.client.worker.worker_interface import WorkerInterface

configuration = Configuration(
    server_api_url=f"http://host.docker.internal:8080/api", debug=True
)

class SillyWorker(WorkerInterface):
    def execute(self, task: Task) -> TaskResult | None:
        return None

    def get_polling_interval_in_seconds(self) -> float:
        return 1

workers = [
    SillyWorker(task_definition_name="silly-worker-1"),
    SillyWorker(task_definition_name="silly-worker-2"),
    SillyWorker(task_definition_name="silly-worker-3"),
    SillyWorker(task_definition_name="silly-worker-4"),
    SillyWorker(task_definition_name="silly-worker-5"),
    SillyWorker(task_definition_name="silly-worker-6"),
    SillyWorker(task_definition_name="silly-worker-7"),
    SillyWorker(task_definition_name="silly-worker-8"),
    SillyWorker(task_definition_name="silly-worker-9"),
    SillyWorker(task_definition_name="silly-worker-10"),
    SillyWorker(task_definition_name="silly-worker-11"),
    SillyWorker(task_definition_name="silly-worker-12"),
]

def start_workers():

    with TaskHandler(workers, configuration) as task_handler:
        task_handler.start_processes()
        task_handler.join_processes()

if __name__ == "__main__":
    start_workers()

Good afternoon. Just want to follow up to let you know I am still doing some testing. So far I have tried with just a generic Session() object and haven't had any memory issues (it stayed around 400MB for a day). I also tried with a generic HTTPAdapter() and it also stayed flat. I am now trying with the Retry() object from urllib3 to see what happens. I am starting with the same setup as the codebase does to see if it reproduced the same issue and then will try playing with these settings to see if there is one setting that's causing this issue. Just wanted to post to keep the thread alive. Hopefully will have some more details very soon.

Good day @c4lm. We tried you suggested along with a few other modifications.
Here are the results:

Creating the session object with the HTTPAdapter and pools set to 1 (connections) and 1 (maxsize). No change in the results and still seeing high memory climbs.

import requests
configuration = Configuration(
server_api_url=f"http://localhost:3500/api", debug=False
)
session = requests.Session()
session.mount('http://', requests.adapters.HTTPAdapter(pool_connections=1, pool_maxsize=1))
configuration.http_connection = session

We took the existing session (from the restclient class) and put it into our session and tweaked the Retry strategy setting total to 1 and backoff_factor to 1 and also removing it altogether. This has no changes to the results. Still getting high memory utilization.

import requests
from urllib3 import Retry
configuration = Configuration(
server_api_url=f"http://localhost:3500/api", debug=False
)
retry_strategy = Retry(
total=1,
backoff_factor=1,
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["HEAD", "GET", "OPTIONS", "DELETE"], # all the methods that are supposed to be idempotent
)
session = requests.Session()
session.mount('http://', requests.adapters.HTTPAdapter(pool_connections=1, pool_maxsize=1, max_retries=retry_strategy))

We also changed the worker to a single worker (removed the other 9) and oddly when we ran memray it produced 14 files instead of 3 (which I would assume it should have created). The images attached are the graphs from each and the states
silly.bin.html

silly.bin.525.html

silly.bin.523.html

silly.bin.521.html

silly.bin.519.html

silly.bin.517.html

silly.bin.515.html

silly.bin.513.html

silly.bin.511.html

silly.bin.509.html

silly.bin.507.html

silly.bin.505.html

silly.bin.503.html

Hi @rydevops do you have some time early next week for a working session on this?  I would like to spend some time to understand the behavior you are seeing, your setup and I would like to replicate the same setup to reproduce.

Hi @rydevops ping!


Explicitly at-mention the user getting the pager

@bswinnerton took a pager from me. I think the Hubot message should've also pinged me in case I was not sitting in the chat.
@technicalpickles @keithduncan What's the appropriate team to ping here?

This isn’t the version of this script used in our hubot install, we depend on hubot-pager-me and will need to make the changes there :+1:

💀


Add match_word argument for git_blacklist

To avoid triggering on add() while dd() is blacklisted. See https://github.com/phpro/grumphp/issues/803

Perhaps it makes sense to change the default value instead of overriding it within the Magento-specific configuration file.
https://github.com/YouweGit/testing-suite/blob/a53b4a1514b134c9a59b248d9471b44af43f5032/config/default/grumphp.yml#L62

@fredden aah yes hadn't seen that there, it totally does!


Restore the note about regen.json

This was removed weeks ago during an update of the README. The code still exists and it still works.
Fixes #156

we could add code to remove regen.json after its used but minimising new code at the moment.

Re-opened here https://github.com/instructlab/instructlab/pull/929


New ASCII logo for Fedora

Description
New ASCII logo added for Fedora was given the approval in this issue https://github.com/dylanaraps/neofetch/issues/1711

the old logo has a small version, what about this one?

I did create a small version for pfetch and neofetch but wasn't sure if i should include it unless you have something better. I didn't get a response about it.

@klpnt I tried making a small version of the logo myself.

Fedora_Small_ASCII.txt
Can you please make a pull request for a small version of the logo using whichever you feel is better?


No accessible initializers (on branch develop-protocol)

On branch develop-protocol, SharingFlowIGOExclusivegram cannot be constructed because it has no accessible initializers.

Add initializer.

Add initializer.


ImportError: No module named 'oauth2client.contrib.locked_file'

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/katsarov/miniconda3/envs/kpi_dashboard/lib/python3.5/site-packages/googleapiclient/discovery_cache/file_cache.py", line 33, in
from oauth2client.contrib.locked_file import LockedFile
ImportError: No module named 'oauth2client.contrib.locked_file'
Traceback (most recent call last):
File "/home/katsarov/miniconda3/envs/kpi_dashboard/lib/python3.5/site-packages/googleapiclient/discovery_cache/init.py", line 41, in autodetect
from . import file_cache
File "/home/katsarov/miniconda3/envs/kpi_dashboard/lib/python3.5/site-packages/googleapiclient/discovery_cache/file_cache.py", line 41, in
'file_cache is unavailable when using oauth2client >= 4.0.0')
ImportError: file_cache is unavailable when using oauth2client >= 4.0.0

@katsar0v this is not en error, just a warning. See #345.

ImportError is warning? Ok

🥇

any remedies?

Any updates?

same

Hi folks, here's some more explanation since there seems to be interest:
This is a warning, that happens to log the exceptions that were raised. The ImportError is being caught and handled (see code snippet below). This function determines what environment the code is being run to choose the correct cache location. Each import will only work in a specific environment.
https://github.com/googleapis/google-api-python-client/blob/66bb32cc70353296dbf1876ae8c22baf6562cfb9/googleapiclient/discovery_cache/__init__.py#L28-L47
If you have comments to make about a closed issue it's best to open a new issue and reference the previous one. Repository maintainers are less likely to see comments on an already closed issue.
Thanks!

Disable cache_discovery
discovery.build(api, version, http=http, cache_discovery=False)
https://stackoverflow.com/questions/40154672/importerror-file-cache-is-unavailable-when-using-python-client-for-google-ser


[AutoPR cosmos-db/resource-manager] [DO NOT MERGE] Cosmos DB Private Link Resources Swagger

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7249

(message created by the CI based on PR content)


Wireframing

Originally created by Tom Smyth at 2015-07-15 02:49, migrated from redmine-#2638

Status changed from New to Closed

By Willie Northway at 2015-09-21 00:15


Reddit text posts copies text instead of linking text posts

Overview
For subreddits such as /r/copypasta where the submissions are all text, have an option to just have Vinny copy the text post and post that.
Affected commands
~tr,~rr copypasta
Expected output
To be fair, you have to have a very high IQ to understand Rick and Morty. The humour is extremely subtle, and without a solid grasp of theoretical physics most of the jokes will go over a typical viewers head. There's also Rick's nihilistic outlook, which is deftly woven into his characterisation- his personal philosophy draws heavily from Narodnaya Volya literature, for instance. The fans understand this stuff; they have the intellectual capacity to truly appreciate the depths of these jokes, to realise that they're not just funny- they say something deep about LIFE. As a consequence people who dislike Rick & Morty truly ARE idiots- of course they wouldn't appreciate, for instance, the humour in Rick's existential catchphrase "Wubba Lubba Dub Dub," which itself is a cryptic reference to Turgenevs Russian epic Fathers and Sons. I'm smirking right now just imagining one of those addlepated simpletons scratching their heads in confusion as Dan Harmon's genius wit unfolds itself on their television screens. What fools.. how I pity them. 😂
And yes, by the way, i DO have a Rick & Morty tattoo. And no, you cannot see it.  Nothin personnel kid 😎
Actual output
https://www.reddit.com/r/copypasta/comments/6qhaln/only_high_iqs_watch_rick_and_morty/
Steps to reproduce
enter that command boi

To be fair I don't know if I have an IQ high enough to enjoy Rick and Morty sauce fully. Therefore I will try my best to fulfill this request.

Closed thanks to @mrb25 in PR #31


Problem with Bootstrap Checkbox

Hello,
I am using library https://github.com/flatlogic/awesome-bootstrap-checkbox for pretty checkboxes.
When I am rendering an image and saving it, then I have some problems with it.

Thank you in advance

@mikeyaremenko
Hello,
I met the same problem.
The real page is like this:

But the screenshot is like this:

My Boostrap version is v3.3.2.
Have you got the solution?
Thanks.

I did not manage to fix that bug with styles, because I decided just to
hide styles for pretty checkboxes before screenshot and return them
afterwards. It is not the best solution but it works.
Allen<EMAIL_ADDRESS>schrieb am Mi., 13. Apr. 2016 um 12:17 Uhr:

@mikeyaremenko https://github.com/mikeyaremenko
Hello,
I met the same problem.
The real page is like this:
[image: real]
https://cloud.githubusercontent.com/assets/678977/14490125/85268c90-01a3-11e6-8958-a58acaffc0f0.png
But the screenshot is like this:
[image: screenshot]
https://cloud.githubusercontent.com/assets/678977/14490136/9611eb62-01a3-11e6-8e58-2299541f13bf.png
My Boostrap version is v3.3.2.
Have you got the solution?
Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/niklasvh/html2canvas/issues/806#issuecomment-209354446


Set all schemes to 'off'

Hi There,
Just wondering whether its possible to turn schemes off after selecting one.
Standard Stata code for similar (trace, off or more, off for instance) doesn't apply here.
Apologies if I've missed this.
regards, Shay

Dear Shay,
There is always one scheme defined. By off, I am assuming you mean reverting back to the default Stata's s2. Here you either need to set it at the beginning of a dofile or you can revert to the default permanently just by typing:
set scheme s2color, perm
You can also select a scheme just for one graph so you are not stuck with it all the time:
scatter y x, scheme(s2color)
Let me know if this helps!
Asjad

Dear Asjad, This is perfect, appreciate the prompt reply.
The schemes are great, just wanted to ensure I could still customise my graphs individually against a background of the Stata default.
Shay


Fix: back button behaviour from bridge page

Description
Fixes this issue which doesn't allow user to go back to previous page from bridge
Current behaviour
Bridge page is stuck in loop when user tries to go back by clicking on browser back button
New behaviour
When user clicks on back button, app navigates user to their previously visited page
Reproducible testing steps:

Go to /bridge?tab=issue/redeem
Navigate to other tabs like transaction or lending
Go back to /bridge page
Navigate using browser back button -> should take you to previous pages in the correct order of stack

@peterslany @danielsimao review please

p.s. @Chanakya888 no need to update the version in package.json—we do that automatically when we prep the releases :)

@Chanakya888 Thanks for your fast contribution!

forgot to mention Peter, I have seen your suggestion to not push the new history to browser stack- I've tried that, it breaks the back flow. If we don't push the exact history /bridge?tab=issue, browser pushes /bridge by default and when users tries to go back /bridge page is empty because it doesn't exist

Actually what I meant in the issue was that we should not push new entry, but replace current instead, but you figured that out, goodjob :))


Fix/python input n parameter args

Related to #57 and #70
This branch contains some modifications and a fix of Parameter.init() and operators.input().

These functions does not take a list no longer in Python.
New function operators.raw_input() takes a list.
Parameter.init() takes only a pair of Shape and Initializer.

Thanks. Fixed.


.Net: Bug: Azure OpenAI client fails with 401 when throttling

When using AzureOpenAITextEmbeddingGenerationService SK client and sending too many requests, the Azure service throttling leads to a "401 Unauthorized" error instead of "429 Too Many Requests".

SK version: 1.20.0
.NET version: 8.0.401

I inspected the HTTP requests and noticed that the internal retry logic is sending a malformed request during retries. It appears to be adding the Authorization header twice (with the same token), e.g.

Request 1 .. 7: ok
Request 8: fails with HTTP Status Code 429
Request 8 is retried with two auth tokens: fails with HTTP Status Code 401

Code to repro:
public static class Program
{
    public static async Task Main()
    {
        var client = new AzureOpenAITextEmbeddingGenerationService(
            endpoint: "https://....openai.azure.com/",
            deploymentName: "text-embedding-ada-002",
            credential: new DefaultAzureCredential()
        );

        for (int i = 0; i < 200; i++)
        {
            Console.WriteLine($"## {i}");
            await client.GenerateEmbeddingsAsync([RndStr(), RndStr(), RndStr(), RndStr(), RndStr(), RndStr(), RndStr()]);
        }
    }

    private static string RndStr()
    {
        var random = new Random();
        return new(Enumerable.Repeat("ABCDEF GHIJKLMNOP QRSTUVW XYZabcdefgh ijklmnopqrst uvwxyz0123 456789", 8000)
            .Select(s => s[random.Next(s.Length)]).ToArray());
    }
}

Output:
## 0
## 1
## 2
## 3
## 4
## 5
## 6
## 7
## 8
Unhandled exception. Microsoft.SemanticKernel.HttpOperationException: Service request failed.
Status: 401 (Unauthorized)

 ---> System.ClientModel.ClientResultException: Service request failed.
Status: 401 (Unauthorized)

   at Azure.AI.OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options)
   at Azure.AI.OpenAI.Embeddings.AzureEmbeddingClient.GenerateEmbeddingsAsync(BinaryContent content, RequestOptions options)
   at OpenAI.Embeddings.EmbeddingClient.GenerateEmbeddingsAsync(IEnumerable`1 inputs, EmbeddingGenerationOptions options, CancellationToken cancellationToken)
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.RunRequestAsync[T](Func`1 request)
   --- End of inner exception stack trace ---
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.RunRequestAsync[T](Func`1 request)
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetEmbeddingsAsync(String targetModel, IList`1 data, Kernel kernel, Nullable`1 dimensions, CancellationToken cancellationToken)
   at Program.Main() in Program.cs:line 21
   at Program.<Main>()

looks like a bug in the underlying Azure.AI.OpenAI 2.0.0-beta.5, I reported it here https://github.com/Azure/azure-sdk-for-net/issues/46109 including a workaround

@RogerBarreto I'm assigning this to you, can you monitor the issue Devis created an pick up the fix when it becomes available.

The original issue was closed, setting this as complete as the original issue is addressed and will be available automatically to Semantic Kernel package in a future release.

@RogerBarreto the outstanding open issue here is the reliance on a pre-release package with a noticeable (quite annoying) bug that impacts all SK consumers.
Rather than waiting for the Azure SDK fix to propagate to SK and SK fix to propagate to consumers, I'd suggest that SK includes a workaround as soon as possible, e.g. by injecting a delegating handler:
internal sealed class AuthFixHandler : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Headers.TryGetValues("Authorization", out var headers) && headers.Count() > 1)
        {
            request.Headers.Authorization = new AuthenticationHeaderValue(
                request.Headers.Authorization!.Scheme,
                request.Headers.Authorization.Parameter);
        }

        return base.SendAsync(request, cancellationToken);
    }
}


RDS Stop Not working - SQL

Hi
I am trying to stop RDS instance, based on certain filters. I see the Lambda runs without error but it does not stops the instance (SQL). Do not understand the logs"[DEBUG] 2018-10-15T22:42:14.920Z 8f663a52-d0cb-11e8-9572-5db93c0eb5c4 metric:ApiCalls Count:0 policy:terminate-unencrypted-public-rds-new restype:rds".
Thanks
Avik

you'll have to produce more of the logs (sanitized, no account numbers) and the policy to help us understand what your seeing. the log line your reporting is an output (only on trunk/master) noting the number of api calls the filters/actions of the policy took.

rds databases can't be stopped for all cases. the rds api only supports stop for databases that are already available. Trying to do this in a lambda on create means the database is still in the 'creating' and is not eligible for stop. I'll look at adding some additional logging around that.

re rds stop limitations see docs https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_StopInstance.html the lifecycle limitations aren't explicitly described, but several others are.

OK i get it now. So i should not be triggering on create DB but something other. Is there are way to trigger the Lambda on create but put it into sleep for time till the instance is up and running?

rds can take a long time to come up fully, there's also no way to correct the problem without alot of manual effort (snapshot, encrypt snapshot, restore, etc), so deleting at create is reasonable since there is no data. else you have to move from execution-mode cloudtrail to config-rule, periodic, or poll with an additional state/status filter on available.

OK, So may be I will try using the action mark-for-op on RDS create and then have a following lambda that would check if instance is running and has a mark-for-op tag and stop those.


Read exported bookmarks from CSV/JSON file

Remember, an issue is not the place to ask questions. If you have issues with the APIs or have a question about the Watson services, see Stack Overflow.
Before you open an issue, please check if a similar issue already exists or has been closed before.
When you open an issue for a feature request, please add as much detail as possible:

[ ] A descriptive title starting with the service name
[ ] A description of the problem you're trying to solve
[ ] A suggested solution if possible

Get the imports to happen from JSON file.


Non-deterministic even with "deterministic=True" "seed=0" and the same number of threads in LightGBM==3.1.1

LightGBM component:
Environment info

Operating System: Linux
CPU/GPU model: CPU
Python version: 3.7.3

LightGBM version or commit hash: 3.1.1, installed by pip
Error message and / or logs

Reproducible example(s)
import lightgbm as lgb
import pandas as pd
import os
import numpy as np
import random
assert lgb.__version__ == '3.1.1'
assert np.__version__ == '1.16.4'
assert pd.__version__ == '1.1.4'
# python 3.7.3
# Linux version 4.15.0-123-generic (buildd@lcy01-amd64-027) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)) #126~16.04.1-Ubuntu

def get_gain_importance(data, label):
    np.random.seed(0)
    random.seed(0)
    os.environ['PYTHONHASHSEED'] = '0'
    gbm = lgb.LGBMClassifier(**{'importance_type': 'gain', 'seed': 0,
                                'deterministic': True, 'n_jobs': n_jobs})
    gbm.fit(data.values, label.values.ravel())
    return gbm.feature_importances_

if __name__ == '__main__':
    path = './'
    n_jobs = 16
    data = pd.read_csv(path+'train_x.csv')
    label = pd.read_csv(path+'train_y.csv')
    FI = get_gain_importance(data, label)
    for i in range(100):
        print(i)
        FI_new = get_gain_importance(data, label)
        if ~(FI == FI_new).all():
            print(FI)
            print(FI_new)
            exit()

data.zip
The data to reproduce is attached in the zip file. Please fill in the "path" and run the code.

@shiyu1994 can you help to check this? it may be related to the bugs you fixed recently.

@shiyu1994 can you help to check this? it may be related to the bugs you fixed recently.

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

Thanks for the rapid response. I will check this tomorrow.

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

Thanks for the rapid response. I will check this tomorrow.

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

It seems that setting force_row_wise=True solves my current issue. Thanks!

@ZhangTP1996 The non-deterministic behavior comes from col-wise and row-wise histogram construction strategy. A quick fix to get a deterministic behavior is to set force_row_wise=True or force_col_wise=True. Otherwise LightGBM automatically chooses col-wise or row-wise strategy using a simple test which can make a different decision in each time.
However, ideally col-wise and row-wise should produce the same result, I'll continue to look into this.

It seems that setting force_row_wise=True solves my current issue. Thanks!

This problem is a numerical issue. Since row-wise and col-wise accumulate gradient values in different ways, when the values of gradients and hessians are quite small, the resultant histogram will have slight differences. Following is the histogram values from the first tree where col_wise and row_wise differ, compared by diff.
6c6
< 0.8558845588,0.1233515594
---
> 0.8558845588,0.1233515668
13c13
< 0.01200347667,0.01187949732
---
> 0.01200347667,0.01187949639
15c15
< 3.043845621e-09,3.048046703e-09
---
> 3.043845399e-09,3.048046481e-09
31c31
< 8.874970808e-05,8.874182822e-05
---
> 8.874967898e-05,8.874180639e-05
38c38
< 0.9603701234,0.038059365
---
> 0.9603701234,0.03805936873
70c70
< -1.129891929e-36,1.129891929e-36
---
> -1.129891839e-36,1.129891839e-36
173c173
< 1.496937505e-38,1.496937505e-38
---
> 1.496937365e-38,1.496937365e-38
224c224
< 0.998657763,0.001340438845
---
> 0.998657763,0.001340438728

With col-wise, bin 31 is selected as threshold, while with row-wise bin 32 is selected.
Considering the numerical problem, I think a quick solution to provide deterministic behavior is to force using row_wise or col_wise when deterministic=True.

Also, I noticed that during training, the training loss vibrates dramatically with the provided data (starting from iteration 38). May be we need to increase the numerical stability.
[1]	valid's binary_logloss: 0.0259893
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[2]	valid's binary_logloss: 0.0308559
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 14
[3]	valid's binary_logloss: 0.0361613
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[4]	valid's binary_logloss: 0.0300254
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[5]	valid's binary_logloss: 0.0287943
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[6]	valid's binary_logloss: 0.0275819
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[7]	valid's binary_logloss: 0.0248224
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[8]	valid's binary_logloss: 0.0252161
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[9]	valid's binary_logloss: 0.0230506
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[10]	valid's binary_logloss: 0.0377827
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[11]	valid's binary_logloss: 0.0263986
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 13
[12]	valid's binary_logloss: 0.0207991
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[13]	valid's binary_logloss: 0.0197117
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[14]	valid's binary_logloss: 0.0192411
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[15]	valid's binary_logloss: 0.018162
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[16]	valid's binary_logloss: 0.0174082
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[17]	valid's binary_logloss: 0.0170077
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[18]	valid's binary_logloss: 0.016684
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[19]	valid's binary_logloss: 0.0191817
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[20]	valid's binary_logloss: 0.0179756
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[21]	valid's binary_logloss: 0.0158201
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[22]	valid's binary_logloss: 0.0193097
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 16
[23]	valid's binary_logloss: 0.014494
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 13
[24]	valid's binary_logloss: 0.019959
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[25]	valid's binary_logloss: 0.0217315
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[26]	valid's binary_logloss: 0.0230094
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[27]	valid's binary_logloss: 0.0206723
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[28]	valid's binary_logloss: 0.0163684
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[29]	valid's binary_logloss: 0.0148431
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[30]	valid's binary_logloss: 0.0289696
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[31]	valid's binary_logloss: 0.0223404
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[32]	valid's binary_logloss: 0.0406943
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 7
[33]	valid's binary_logloss: 0.0394845
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[34]	valid's binary_logloss: 0.0388705
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[35]	valid's binary_logloss: 0.0758646
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[36]	valid's binary_logloss: 0.0532624
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 12
[37]	valid's binary_logloss: 0.0820735
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[38]	valid's binary_logloss: 0.0800012
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[39]	valid's binary_logloss: 0.186073
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[40]	valid's binary_logloss: 0.0959558
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[41]	valid's binary_logloss: 0.114168
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[42]	valid's binary_logloss: 0.145749
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 10
[43]	valid's binary_logloss: 0.153056
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[44]	valid's binary_logloss: 0.101863
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 8
[45]	valid's binary_logloss: 0.021308
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[46]	valid's binary_logloss: 0.149215
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[47]	valid's binary_logloss: 0.0238757
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 9
[48]	valid's binary_logloss: 0.107908
[LightGBM] [Debug] Trained a tree with leaves = 31 and max_depth = 11
[49]	valid's binary_logloss: 0.0173251

Closed via #4027.


Zenarmor plugin can't discover interfaces when development deployment type is selected

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[X] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
[X] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
[X] The title contains the plugin to which this issue belongs

Describe the bug
Zenarmor cannot discover interfaces when "Development" deployment type is selected.
The "wheel" keeps going. After changing to "Production" type - works as expected.
Screenshots attached.
Environment
Versions OPNsense 23.1_6-amd64
FreeBSD 13.1-RELEASE-p5
OpenSSL 1.1.1s 1 Nov 2022
Proxmox VM on HP T630

PHP 8.1 has become pretty strict. Probably an warning being emitted in backend code breaking JSON format or so... CC @muratbalaban43

@fichtner @szafranski , looking at it.


RR-143 implement the DPS connect Footer component

Description
Implement the DPS connect Footer component and CSS/JS assets.
Tickets
https://dsdmoj.atlassian.net/browse/RR-143
https://dsdmoj.atlassian.net/browse/RR-295
https://dsdmoj.atlassian.net/browse/RR-296
https://dsdmoj.atlassian.net/browse/RR-297
Screenshots

Also, does this PR handle the eventuality that we the feature toggle is enabled, but the API call fails? In that case we need to display the approved Connect DPS fallback, which I believe is a simple grey box.
In the ticket there are 3 ACs - I'm not sure this PR is meeting the last one yet?

Maybe not for this PR, but as part of this story (create a new subtask for it) we will need to

enable the feature toggle in feature.env for our cypress integration tests
create a wiremock stub for the cypress tests so that every cypress test gets the footer from the API
create a wiremock stub in wiremock/mappings to support running it locally with wiremockj

Created ticket for this https://dsdmoj.atlassian.net/browse/RR-316

Also, does this PR handle the eventuality that we the feature toggle is enabled, but the API call fails? In that case we need to display the approved Connect DPS fallback, which I believe is a simple grey box. In the ticket there are 3 ACs - I'm not sure this PR is meeting the last one yet?

Added this as part of the to do items for tomorrow!

Also, does this PR handle the eventuality that we the feature toggle is enabled, but the API call fails? In that case we need to display the approved Connect DPS fallback, which I believe is a simple grey box. In the ticket there are 3 ACs - I'm not sure this PR is meeting the last one yet?

Tested this and the footer falls back to the standard footer we currently have, the same one that DPS provide the templates for is the just the basic Nunjucks Footer!

Added the third footer scenario fallback

Feature toggle disabled - show the footer we always have done
Feature toggle enabled and API call works - show the data from the API as the footer
Feature toggle enabled and API call fails - show the empty grey footer

Nice 👍 LGTM - as mentioned in a couple of comments I think some of the error handling might need a little looking at / tidying, but I'm happy to do that after you've merged this 👍

I've created the ticket https://dsdmoj.atlassian.net/browse/RR-318


excessive memory use during build kills Travis CI jobs

The 2.12 / scalaz-7.2 Travis builds are intermittently failing with:
bin/travis: line 49:  2089 Killed                  sbt 'set scalazVersion in ThisBuild := System.getenv("SCALAZ_VERSION")' ++$TRAVIS_SCALA_VERSION -jvm-opts "$JVM_OPTS_FILE" $SBT_COMMAND
The command "bash bin/travis" exited with 137.

Suspect our builds are exceeding the memory allocated to the Travis Docker containers and being picked off by the kernel OOM killer.

One example was build 3726.6 (before I realized it was the start of a pattern and restarted it).   Your build exited with 1.

Another example is 3727.6.  It died further along:
[info] Packaging /home/travis/build/http4s/http4s/scala-xml/target/scala-2.12/http4s-scala-xml_2.12-0.16.0a-SNAPSHOT.jar ...
[info] Done packaging.
[info] 	published http4s-core_2.12 to https://oss.sonatype.org/content/repositories/snapshots/org/http4s/http4s-core_2.12/0.16.0a-SNAPSHOT/http4s-core_2.12-0.16.0a-SNAPSHOT-sources.jar
[info] Done packaging.
[warn] /home/travis/build/http4s/http4s/client/src/main/scala/org/http4s/client/Client.scala:197: method run in class Task is deprecated (since 7.2): use unsafePerformSync

...

[warn] /home/travis/build/http4s/http4s/client/src/main/scala/org/http4s/client/impl/RequestGenerator.scala:19: Could not find any member to link for "org.http4s.Request".
[warn]   /** Make a [[org.http4s.Request]] using this Method */
[warn]   ^
model contains 6 documentable templates
model contains 5 documentable templates
[warn] /home/travis/build/http4s/http4s/client/src/main/scala/org/http4s/client/Connection.scala:19: Could not find any member to link for "Response".
[warn]   /** Close down the connection, freeing resources and potentially aborting a [[Response]] */
[warn]   ^
[warn] 6 warnings found
[info] Main Scala API documentation successful.
[info] Packaging /home/travis/build/http4s/http4s/client/target/scala-2.12/http4s-client_2.12-0.16.0a-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
model contains 6 documentable templates
bin/travis: line 49:  2065 Killed                  sbt 'set scalazVersion in ThisBuild := System.getenv("SCALAZ_VERSION")' ++$TRAVIS_SCALA_VERSION -jvm-opts "$JVM_OPTS_FILE" $SBT_COMMAND

The command "bash bin/travis" exited with 137.
store build cache
Done. Your build exited with 1.

The first time I saw this was on #950, which lines up with the 2.12 observation.  This could be new in TLS 2.12.1.

For future reference, the failed build on merge of #950 into cats was 3705.1.

Probably not TLS (Typelevel Scala compiler) as TLS is only used on the cats branch.
Alternative theory is growth in metaspace memory usage exceeding the Travis 4gb limit on its container infrastructure.
As mentioned in gitter:

Wild guess on #965: The build uses all 2gb of allowed heap, MaxMetaspaceSize is not set, and we recently crossed a threshold where up to 2gb of metaspace can be used. Once we hit the 4gb mark (cap on Travis container builds), the OOM killer zaps us.

Sherlock Holmes! Glad @bfritz Is taking such good care of the CI


IOS map doesnt render

I'm submitting a ...  (check one with "x")

[ ] question
[X] any problem or bug report

OS:  (check one with "x")

[ ] Android
[X] iOS

cordova information:  (run $> cordova plugin list)
com-sarriaroman-photoviewer 1.1.17 "PhotoViewer"
com.googlemaps.ios 2.7.0 "Google Maps SDK for iOS"
cordova-plugin-actionsheet 2.3.3 "ActionSheet"
cordova-plugin-app-version 0.1.9 "AppVersion"
cordova-plugin-camera 4.0.3 "Camera"
cordova-plugin-device 2.0.2 "Device"
cordova-plugin-dialogs 2.0.1 "Notification"
cordova-plugin-email-composer 0.8.15 "EmailComposer"
cordova-plugin-file 6.0.1 "File"
cordova-plugin-file-transfer 1.7.1 "File Transfer"
cordova-plugin-google-analytics 1.8.3 "Google Universal Analytics Plugin"
cordova-plugin-googlemaps 2.3.8 "cordova-plugin-googlemaps"
cordova-plugin-inappbrowser 3.0.0 "InAppBrowser"
cordova-plugin-ionic-keyboard 2.0.5 "cordova-plugin-ionic-keyboard"
cordova-plugin-ionic-webview 1.1.19 "cordova-plugin-ionic-webview"
cordova-plugin-network-information 2.0.1 "Network Information"
cordova-plugin-splashscreen 5.0.2 "Splashscreen"
cordova-plugin-statusbar 2.4.2 "StatusBar"
cordova-plugin-whitelist 1.3.3 "Whitelist"
cordova-plugin-x-socialsharing 5.4.0 "SocialSharing"
cordova-sqlite-storage 2.3.2 "Cordova sqlite storage plugin"
es6-promise-plugin 4.2.2 "Promise"
mx.ferreyra.callnumber 0.0.2 "Cordova Call Number Plugin"
phonegap-plugin-push 2.2.3 "PushPlugin"
uk.co.workingedge.phonegap.plugin.launchnavigator 4.2.1 "Launch Navigator"

If you use @ionic-native/google-maps, please tell the package.json (only @ionic-native/core and @ionic-native/google-maps are fine mostly)
@ionic-native/core : ^4.7.0
@ionic-native/google-maps : ^4.8.2

Current behavior:
On iPhone X the map loads up with no bugs and displays my markers and shows the google icon but the map itself does not display.
Expected behavior:
The map should render the world,
Screen capture or video record:

Related code, data or error log (please format your code or data):
CoreData: annotation:  Failed to load optimized model at path '/var/containers/Bundle/Application/3191DA34-0C0C-40EB-97AF-9CDBAC124DE8/Modern Marketing Coach.app/GoogleMaps.bundle/GMSCacheStorage.momd/StorageWithTileProto.omo'
CoreData: annotation:  Failed to load optimized model at path '/var/containers/Bundle/Application/3191DA34-0C0C-40EB-97AF-9CDBAC124DE8/Modern Marketing Coach.app/GoogleMaps.bundle/GMSCacheStorage.momd/StorageWithTileProto.omo'
CoreData: annotation:  Failed to load optimized model at path '/var/containers/Bundle/Application/3191DA34-0C0C-40EB-97AF-9CDBAC124DE8/Modern Marketing Coach.app/GoogleMaps.bundle/GMSCacheStorage.momd/StorageWithTileProto.omo'

Its relating to this issue but these solutions dont work

https://stackoverflow.com/questions/47236769/failed-to-load-optimized-model-googlemaps-sdk-ios

Please share your code

If you are talking about gray map, check out here.
https://github.com/mapsplugin/cordova-plugin-googlemaps-doc/blob/master/troubleshootings/graymap/README.md

That link resolved my issue thanks


Verify that HTTPS tunnel blocking does not mess with Google services

See details here:
https://github.com/AdguardTeam/AdguardForAndroid/issues/1334

Some users responded that AdGuard will make Google Maps and the backup to Google Photos abnormal.
Without AdGuard protection, it can normally start to do the backup to Google Photos after connecting to Wi-Fi.
With AdGuard protection, the backup to Google Photos is abnormal.
It will show "wait for Wi-Fi connection" even if actually we have connected to Wi-Fi.


Encode result output to utf-8 before passing to subunit

This must be bytes under python3.

Not sure what sort of testing you usually do. I just ran it against some tests from another project.

Hey, thanks for the patch. I don't use this project anymore I currently don't write python code :(
The change seems to be ok so I will merge it.
Previously I was testing it with my python project which is not maintained anymore.


Publish 3.0.0 to NPM

Also, add npm version badge and remove version from title.

Fixed in https://github.com/taataa/taaspace/commit/b6f9eb5d3caa385e11521afba48f3eef8850f634


chore add submodules in .idea/vcs.xml

Proposed changes
Add submodules from be in .idea/vcs.xml.
Also Sort XML tags in alphabetical order.
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Added patch for CVE-2023-46136 to python-werkzeug

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[x] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[x] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[x] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[x] Ready to merge

Summary

Upgrades and adds patch to python-werkzeug to fix CVE-2023-46136
Change Log
Upgrades python-werkzeug to 2.3.7
Patches CVE-2023-46136 in python-werkzeug
Does this affect the toolchain?

NO
Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-2023-46136

Test Methodology

Pipeline build id: 451961 Unified-Buddy-Build

@nisamson, just a heads-up: all PRs against fasttrack/2.0 should be rebased, since we finished a monthly update, which also updated fasttrack/2.0. Rebasing (or merging) the latest state of the branch will re-trigger the PR check and allow us to validate that the latest state of the repo still builds correctly with the changes from the PRs.

Auto cherry-pick results:

main :white_check_mark: -> https://github.com/microsoft/CBL-Mariner/pull/6802

Auto cherry-pick pipeline run -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=455727&view=results


little-snitch: update zap

Updated to match developer documentation:
https://www.obdev.at/support/index.html?product=LS&topic=faq&entry=245442241039726

[x] Checked there aren’t open pull requests for the same cask.
[x] Checked there aren’t closed issues where that cask was already refused.
[x] When naming the cask, followed the token reference.
[x] Commit message includes cask’s name.
[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] brew cask install {{cask_file}} worked successfully.
[x] brew cask uninstall {{cask_file}} worked successfully.

Uninstall for this cask needs some work. We basically need the equivalent of the 'installer manual:' stanza so that when 'brew uninstall little-snitch' completes the user is notified that '/Library/Little Snitch/Little Snitch Uninstaller.app' will need to be run manually and the computer restarted to complete the install.
Please advise the best way to accomplish this?

Well, that’s the problem, we don’t have it.
We’re aware this cask is missing uninstall. We don’t allow any more casks without it, but this one was added before the rule and the app is popular.
Presumably whatever the cask is doing with its uninstallation can be replicated with CLI commands. Issue is we don’t know what the uninstall is really doing. Best way to go about it would be to contact the developers and ask how to perform the uninstallation, or better yet, if they would mind providing a script we could run (it has happened before).
I’ll still merge this PR now since you are still making a useful change, and the cask was already present.


Not working on windows

VScode info:
Version: 1.92.2 (user setup)
Commit: fee1edb8d6d72a0ddff41e5f71a671c23ed924b9
Date: 2024-08-14T17:29:30.058Z
Electron: 30.1.2
ElectronBuildId: 9870757
Chromium: 124.0.6367.243
Node.js: 20.14.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22631
Issue
Extension won't start
Probable cause
It fails to open any file as workspacePath is incorrectly defined.
Possible solution
in file src/extension.ts at line 23
replace
const workspacePath = workspace.workspaceFolders![0].uri.path;

with
const workspacePath = workspace.workspaceFolders![0].uri.fsPath;

Tested on system above and works, not tested on any linux disto
Thanks a lot for you work, it's very useful

Thank you for your issue report @L-Carloni


Hugo doesn't support Markdown tables

I pasted the RippleDoc Quick Markdown Example source from this page:
http://www.unexpected-vortices.com/sw/rippledoc/quick-markdown-example.html
into a new Hugo post. I tried the "hugoscrolls" and "hugo-uno" themes, and in both cases the tables did not render at all.
I'm using the latest released Hugo v0.21 under FreeBSD 64-bit.

Hugo uses blackfriday to render the Markdown. As you can see tables are supported. The difference to your linked example is that columns have to be separated by using the | symbol.
But refer for such questions to the Hugo forum; GitHub is used for concrete issues and requests/enhancements only.

Apologies, and thank you for the help.


fix: Unable to successfully post more than 1 social media post

Try posting multiple social media posts

Came across an issue when testing this ticket.
I already had my twitter account connected in my local env, when I tried to post to twitter I came across the below error saying I was not authenticated. (When I go to the social media settings page it still says I am logged in and that the link to my account expires never)

I removed and re-added my twitter account. Then when I try to post to twitter it successfully posts.

I'm not sure if this was an issue with the work completed in this ticket or if I've encountered a bug. Will ask someone else from the team to check if they can repeat this behaviour.

Came across an issue when testing this ticket.
When I tried posting two twitter posts I came across the below error saying I was not authenticated. (When I go to the social media settings page it still says I am logged in and that the link to my account expires never)
I removed and re-added my twitter account. Then when I try to post to twitter it successfully posts.
I'm not sure if this was an issue with the work completed in this ticket or if I've encountered a bug. Will ask someone else from the team to check if they can repeat this behaviour.
Otherwise I have testing posting two posts at the same time (hootsuite and twitter) and it works as expected.

Not related to this ticket, will monitor and raise bug ticket if it happens again.


Update SA1302 (InterfaceNamesMustBeginWithI) to handle conflicting names

The following code results in an SA1302 warning. If the code fix is applied, it introduces a new compiler error.
public interface Foo { }
public interface IFoo { }
public class Bar : Foo { }

We first need to decide how we want to handle this case (and similar), followed by implementation and tests.

First of all we need to check if a type with the new name exists in the same namespace. If it's in a different namespace - it's safe to apply the fix. If the type aleady exists in the namespace, maybe we should append a number to the name, to make it unique  (of course, if we have Foo interface, and IFoo and IFoo2  already exist, we should rename it to IFoo3).

We could do that. First I'm curious if there is a way to check with Renamer to see if the rename operation introduced new errors, and cancel the code fix if it did.

Isn't it done in #443 ?

No, that pull request makes improvements based on the feedback in dotnet/roslyn#197, but this particular issue did not behave in the manner expected by that team. Hopefully the next release of the code fix framework allows us to notify the user of a conflict prior to applying it.


Cannot delete element from _varchar with value editor

System information:

Operating system (distribution) and version: Ubuntu 20.04
DBeaver version: 7.2.2
Additional extensions: None

Connection specification:

Database name and version: PostgreSQL 12
Driver name: PostgreSQL
Do you use tunnels or proxies (SSH, SOCKS, etc)?  No

Describe the problem you're observing:
I cannot delete an element from a _varchar array using the value editor.
Steps to reproduce, if exist:

Create a test table: CREATE TABLE domains (id_domain serial NOT NULL, urls _varchar NULL);
Insert a row: INSERT INTO domains VALUES (1, '{"http://www.google.com","https://www.google.com"}');
Using the value editor, try to remove an element from the array.

When you save it, you will get the following error.

Include any warning/errors/backtraces from the logs
Error creating struct
  Unable to find server array type for provided name _varchar.

thanks for the bug report

I have a similar problem but not exacly the same. I would like to know if i must to open new issue for that. I Cannot edit any record from editor, the answer is:
Error synchronizing data with database
Reason:
Object parameter [79c0b4c2-5ca0-4e2c-b6cb-db87a9467d8e] binding not supported

I have the same bug. After creating a row, I can't add, edit values or delete them. It says:
_Error creating struct
Unable to find server array type for provided name varchar.

not reproducible in 7.2.3

i am sorry but, why this is closed?
Postgresql
dbeaver Version <IP_ADDRESS>010181902
Button Save reproduces this error:

Thank you

@SephShom your error message looks very different from the one of this issue. #10048 should be your bug.

ok thank you. That s why in my first message i asked if i'd need to open a
new issue. I will, thank you.
On Sun, Oct 25, 2020 at 2:43 PM Paolo Inaudi<EMAIL_ADDRESS>wrote:

@SephShom https://github.com/SephShom your error message looks very
different from the one of this issue. #10048
https://github.com/dbeaver/dbeaver/issues/10048 should be your bug.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dbeaver/dbeaver/issues/9991#issuecomment-716192453,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFKKLNDDENUYZSBKJIVLEPTSMRWW5ANCNFSM4SGKFU2Q
.


crash when leaving a rf tools dim(1.12)

Forge:1.12-2410
RF dim: rftoolsdim-1.12-5.02
RF tools: rftools-1.12-6.10beta4
Crash report:
crash-2017-07-14_20.16.39-server.txt

I'll check it out

Oh, but this is already fixed in latest version


xemu: 0.7.123 -> 0.7.127

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/xemu/versions.
meta.description for xemu is: Original Xbox emulator
meta.homepage for xemu is: https://xemu.app/
meta.changelog for xemu is: https://github.com/xemu-project/xemu/releases/tag/v0.7.127
Updates performed

Version update

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.7.127 with grep in /nix/store/s7mxa60pc76zbqq6blrgqa4s51w2d7n4-xemu-0.7.127
found 0.7.127 in filename of file in /nix/store/s7mxa60pc76zbqq6blrgqa4s51w2d7n4-xemu-0.7.127

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

xemu

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/s7mxa60pc76zbqq6blrgqa4s51w2d7n4-xemu-0.7.127 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A xemu https://github.com/r-ryantm/nixpkgs/archive/837a5b73808158669ee48cdec20c1c7c13e361f7.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/837a5b73808158669ee48cdec20c1c7c13e361f7#xemu

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/s7mxa60pc76zbqq6blrgqa4s51w2d7n4-xemu-0.7.127
ls -la /nix/store/s7mxa60pc76zbqq6blrgqa4s51w2d7n4-xemu-0.7.127/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

2 packages built:

xemu
xemu.debug

Maintainer pings
cc @AndersonTorres for testing.

@NixOS/nixpkgs-merge-bot merge


viewBox is missing from queercat_*.svg files

Some of the queercats have their viewBox attribute missing, which prevents proper scaling:

I think 1baa3eedb2b643ae35e574752e811d3c1efbc8d9 is to blame - whatever optimiser you were using stripped out viewBox too :crying_cat_face:
Manually adding viewBox="0 0 750 750" fixes the problem.

Thanks for the comment! I’ll see these get added back in.
[ ] Add viewBox to all the things


How to capture a mouse click on a page with an image and draw a dot based on that captured location?

What should be the thought process or code to capture the position of a mouse with respect to an image loaded on the window and then paint/draw a dot at that location?

So, I would put Image control into Container and added on_click handler to container. In event's data you will receive a string with both local (relative to container position) and global (relative to app window) mouse pointer coordinates in the format local_x:local_y global_x:global_y. Then, Container could be put into Stack and finally dots could be containers with round corners positioned absolutely inside Stack. Let me know if you need a code example :)

Thanks! Will try this approach when we get the time and get back to you if we need more help. Closing this issue for now.

I'd like to leave a note: if ink is enabled, event data will be empty.(version: 0.1.59) This is feature, not bug.
I spent 30 minutes on this, hope this note can help others!


svelte a11y warning "A11y: on:mouseover must be accompanied by on:focus" in version 3.39

I have this too.

Fixed by https://github.com/IBM/carbon-components-svelte/commit/a67e8a81f6dc5d7416ed6f4468efeb759593873c and released in v0.42.1


Issue with HTTP call formatting in DaprClient

I am using DaprClient for HTTP calls.
I'm having an issue with the format of the response when getting the string of the response via a get request.
My RPC (my-app):
[HttpGet]
public IActionResult Test()
{
    return Ok("my string text");
}
DaprClient Invoke 1:
string str = await daprClient.InvokeMethodAsync<string>(HttpMethod.Get, "my-app", "test");
I got "Dapr.Client.InvocationException" exception:
An exception occurred while invoking method: my-app on app-id: test.
And the InnerException "System.Text.Json.JsonException":
\u00270xE6\u0027 is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
DaprClient Invoke 2:
Next, I tried a more low-level call:
var req = daprClient.CreateInvokeMethodRequest(HttpMethod.Get, "my-app", "test");
var res = await daprClient.InvokeMethodWithResponseAsync(req);
return Ok(new
{
    a = res.StatusCode,
    b = await res.Content.ReadAsStringAsync(),
});
Then I got the result:
{
	"a": 200,
	"b": "my string text"
}
It can be seen that it responds in "text/plain" format. If it is "application/json", the value of b will be "\"my string text\"".
Summarize:
I looked at the source code for "CreateInvokeMethodRequest" in "DaprClientGrpc.cs" and none of the HTTP calls explicitly specify "Accept" (eg: application/json). However, JSON serialization is declared in the annotation of the generic method. So not specifying the Accept header, is it "incomplete"? In the case of "my-app", it responds in "text/plain" format by default, which makes JSON formatting impossible.

I think the comment in the source code mentions "The type of the data that will be JSON deserialized from the response body" is ambiguous, which would mean that the SDK internally defines the "JSON" format and then doesn't actually specify "Accept" .
Please confirm.

I don't believe we specify any content headers, though you could yourself by providing an appropriate httpclient/factory to the client construction.
InvokeMethodAsync<TResponse>() does try and deserialize whatever the response is into the given object. Since you aren't passing an object back, but rather a raw string, the serialization fails.


Fix use of 'long' in clickstream.

'Long' is not a valid data type for KSQL. Switching to 'BigInt'.

BTW, why do we have two copies of the clickstream demo? One each in KSQL and quickstart-demo repos?
https://github.com/confluentinc/quickstart-demos/pull/41

I've updated the SQL file inline with the KSQL repo version. This included dropping one of the statements, as it was causing an error:

Windowing not supported run script 'ksql-clickstream-demo/demo/clickstream-schema.sql';for table aggregations

BTW, why do we have two copies of the clickstream demo? One each in KSQL and quickstart-demo repos?

@big-andy-coates : the reason is that the one in KSQL is Docker only and the one in quickstart-demo is local install only
To avoid copying edits in two different places, we have some options:
Option 1: start.sh script does a git clone of ksql repo to get clickstream demo code
Option 2: move clickstream demo (Docker version) from ksql to quickstart-demos repo
Open to discussion.  cc: @rmoff

I'd go with option 1, as the docker images are built of the Ksql repo, so we'd then only have one copy of things, where as if you were to go with option 2 we'd still need a copy of things in the KSQL repo to build the docker images from, and there'd be another copy in this repo.
Also, keeping it all in KSQL means its much more likely engineers with fix up the SQL when/if changing things as they may be find it when searching the repo for other uses.
BTW: the KSQL repo still covers non-docker too: https://github.com/confluentinc/ksql/blob/9d39ff8a74e237389bf70e6ca1fcca136c13445e/ksql-clickstream-demo/non-docker-clickstream.md

Oh, and also, I don't have rights to merge into this repo. So you'll either need to give me permission or merge this for me :D

@rmoff , do you have thoughts on https://github.com/confluentinc/quickstart-demos/pull/41#issuecomment-422086438 ?

@rmoff , do you have thoughts on #41 (comment) ?

Yes, detailed in https://confluentinc.atlassian.net/browse/DEVX-289

@rmoff : please note my earlier comment about the branch into which you want to merge.  Right now the PR is set to merge into master, which is not validated and will not propagate to the default branch users get when they clone.  You need to reset to the appropriate branch (5.0.0-post?  5.0.1-post?) and rebase your code if needed

I'm going to close this PR as I'm going to fix the issue as part of migrating code all into a single place.


Performance: Targets inheriting from NLog.Targets.TargetWithLayout can compile layout excessively

NLog version: 4.7.2
Platform: .Net 3.5 / .Net 4.5 / .NET Core 2
If one sets up the Layout property on a target inheriting from NLog.Targets.TargetWithLayout, LayoutParser.CompileLayout will be invoked at least twice: once for the default value set inside the constructor of NLog.Targets.TargetWithLayout, and the second time for the value set by the user.
Expected behavior:
Only compile layout once and only for the value which is going to be used.
P.S. I understand it is pretty minor, but if that is not too much effort...

I understand it is pretty minor, but if that is not too much effort...

You are welcome to create a PullRequest for the NLog-project, and show your ideas.

Closing as inactive.


Installer stops after creating directory

I've been trying to get everything installed, but the installation script dies right after creating the project.  I've tried deleting all the docker volumes, uninstalling and reinstalling docker, but no change.  I'm on an M1 Mac with V 4.3.2 of docker desktop.

composer create-project nystudio107/craft:dev-craft-vite testvite --no-install --remove-vcs

Creating a "nystudio107/craft:dev-craft-vite" project at "./testvite"
Installing nystudio107/craft (dev-craft-vite 87fcc798f235eb170b395f6e019fb06cb9e93997)

Installing nystudio107/craft (dev-craft-vite 87fcc79): Extracting archive
Created project in /Users/lyle/Sites/testvite

@php -r "copy('./cms/example.env', './cms/.env');"
@php -r "copy('./scripts/example.env.sh', './scripts/.env.sh');"
@php -r "@unlink('composer.json');"
@php -r "@unlink('composer.lock');"
@php -r "@unlink('LICENSE.md');"
@php -r "@unlink('README.md');"

Any help would be appreciated!  Thanks!

I think you're doing it right. Now you'll need to cd into that new directory with cd testvite. From there you should be able to get things going by typing make dev in that directory.
The output that you shared above is just showing you that it successfully created that new project from the composer template.