Support for "_id" for pull replication.

While doing pull replication for all documents, we are not able to replicate "_id" parameter, it comes as  "nil".
But when a document comes inside a document the "_id" parameter is coming fine.

Hi @SakshiJainAirfi, could you please share code that reproduces the issue? Also, could you provide a sample document(s) that you are trying to replicate? And, what version of the library are you using?

Hi @SakshiJainAirfi, could you please share code that reproduces the issue? Also, could you provide a sample document(s) that you are trying to replicate? And, what version of the library are you using?

Hi emlaver,
CDTDatastore  -> 2.1.1
I have base Model class for mapping products and same class is inherited by Combo model class.
Both the model class has property named as "_id". When both the classes try to parse "_id" it is parsed as nil.
But when inside Combo class Product Model class parse "_id" then it come as proper string.
Attached are screenshots of parsed object.

@peetersn

Hi emlaver,
CDTDatastore  -> 2.1.1
I have base Model class for mapping products and same class is inherited by Combo model class.
Both the model class has property named as "_id". When both the classes try to parse "_id" it is parsed as nil.
But when inside Combo class Product Model class parse "_id" then it come as proper string.
Attached are screenshots of parsed object.

@peetersn

@SakshiJainAirfi Thanks for sharing additional info.  In your first screenshot above you have a red circle around id field that points to vendorId.  Are you expecting that the vendorId has the same value as id?  Which model class are these fields (including id and vendorId) in?
I'm unable to see the second screenshot, could you upload again? Also, can you share your base, Combo , and Product model classes? Just to clarify, the Product model is nested within the Combo model class?

@SakshiJainAirfi Thanks for sharing additional info.  In your first screenshot above you have a red circle around id field that points to vendorId.  Are you expecting that the vendorId has the same value as id?  Which model class are these fields (including id and vendorId) in?
I'm unable to see the second screenshot, could you upload again? Also, can you share your base, Combo , and Product model classes? Just to clarify, the Product model is nested within the Combo model class?

@emlaver I have updated screenshots. I don't have any concern for vendorId. Only concern is for " _id".

@emlaver I have updated screenshots. I don't have any concern for vendorId. Only concern is for " _id".

@SakshiJainAirfi how did you retrieve the document from the local store? the library has two ways for retrieving replicated documents: 1) retrieve a CDTDocumentRevision by passing _id or 2) get documents by querying.

@SakshiJainAirfi how did you retrieve the document from the local store? the library has two ways for retrieving replicated documents: 1) retrieve a CDTDocumentRevision by passing _id or 2) get documents by querying.

we are getting by query
    public func getProduct(iataCode: String, productId: String) -> Product? {
        let name = String(describing: Product.self)
        let query = [KEY_BY_IATA_CODE: iataCode, KEY_BY_TYPE: name, KEY_BY_PRODUCT_ID: productId]
        return getDocument(query: query)

we are getting by query
    public func getProduct(iataCode: String, productId: String) -> Product? {
        let name = String(describing: Product.self)
        let query = [KEY_BY_IATA_CODE: iataCode, KEY_BY_TYPE: name, KEY_BY_PRODUCT_ID: productId]
        return getDocument(query: query)

@SakshiJainAirfi thanks for sharing the code snippet.  Could you also share the code in your function getDocument?

@SakshiJainAirfi thanks for sharing the code snippet.  Could you also share the code in your function getDocument?

public func getDocuments<T: Codable>(query: JSONDict) -> [T] {
        objc_sync_enter(CloudantBaseDao.mutex)
        defer {
            objc_sync_exit(CloudantBaseDao.mutex)
        }

        Log.debug("\(T.self) DAO fetching \(T.self) with query: \(query)")
        self.dataStore?.updateAllIndexes()
        var documents: [T] = []
        let resultSet: CDTQResultSet? = getDocumentByRule(query: query)

        guard let result = resultSet, result.documentIds.count > 0 else {
            Log.error("Failed to find Documents For \(T.self)")
            return documents
        }

        result.enumerateObjects { [weak self] (doc, idx, stop) -> Void in
            autoreleasepool {
                do {
                    let data: Data = try doc.documentAsDataError()
                    // Decode data to object
                    let jsonDecoder = JSONDecoder()
                    let document: T? = try jsonDecoder.decode(T.self, from: data)
                    if document != nil {
                        documents.append(document!)
                    }
                } catch {
                    let error = NSError(domain: "CloudantBaseDao", code: 404, userInfo: [NSLocalizedDescriptionKey:
                        error.localizedDescription, "DB": self?.databaseName ?? "", "Device ID": AppGlobals.deviceID, "Query": query])
                    Crashlytics.crashlytics().record(error: error)
                    Log.error("CloudantBaseDao Error: \(error), Document ID: \(doc.docId ?? "No ID Found")")
                }
            }
        }

        return documents
    }

public func getDocuments<T: Codable>(query: JSONDict) -> [T] {
        objc_sync_enter(CloudantBaseDao.mutex)
        defer {
            objc_sync_exit(CloudantBaseDao.mutex)
        }

        Log.debug("\(T.self) DAO fetching \(T.self) with query: \(query)")
        self.dataStore?.updateAllIndexes()
        var documents: [T] = []
        let resultSet: CDTQResultSet? = getDocumentByRule(query: query)

        guard let result = resultSet, result.documentIds.count > 0 else {
            Log.error("Failed to find Documents For \(T.self)")
            return documents
        }

        result.enumerateObjects { [weak self] (doc, idx, stop) -> Void in
            autoreleasepool {
                do {
                    let data: Data = try doc.documentAsDataError()
                    // Decode data to object
                    let jsonDecoder = JSONDecoder()
                    let document: T? = try jsonDecoder.decode(T.self, from: data)
                    if document != nil {
                        documents.append(document!)
                    }
                } catch {
                    let error = NSError(domain: "CloudantBaseDao", code: 404, userInfo: [NSLocalizedDescriptionKey:
                        error.localizedDescription, "DB": self?.databaseName ?? "", "Device ID": AppGlobals.deviceID, "Query": query])
                    Crashlytics.crashlytics().record(error: error)
                    Log.error("CloudantBaseDao Error: \(error), Document ID: \(doc.docId ?? "No ID Found")")
                }
            }
        }

        return documents
    }

@SakshiJainAirfi in your enumeration block you call documentAsDataError on the current document.  The documentAsDataError does not return document metadata such as _id or _rev.  You'll want to add these properties manually to your document object: after line let document: T? = try jsonDecoder.decode(T.self, from: data):
...
let document: T? = try jsonDecoder.decode(T.self, from: data)
document.id = doc.docId // add doc _id
...

@SakshiJainAirfi in your enumeration block you call documentAsDataError on the current document.  The documentAsDataError does not return document metadata such as _id or _rev.  You'll want to add these properties manually to your document object: after line let document: T? = try jsonDecoder.decode(T.self, from: data):
...
let document: T? = try jsonDecoder.decode(T.self, from: data)
document.id = doc.docId // add doc _id
...

@emlaver How I can add "_id" manually, this document am parsing from server.
One more point I have observed, when I see local database I can’t see "_id" for higher level nodes for inside nodes "_id" is coming fine (as shown In above screenshot).

@emlaver How I can add "_id" manually, this document am parsing from server.
One more point I have observed, when I see local database I can’t see "_id" for higher level nodes for inside nodes "_id" is coming fine (as shown In above screenshot).

When you do a pull replication the documents from the server are inserted into the local data store.
When you query the local data store matching documents are returned as CDTDocumentRevision instances.
So the variable doc you are emumerating in result.enumerateObjects are instances of CDTDocumentRevision from the local datastore.
In a CDTDocumentRevision the root level _ prefixed document metadata is processed separately from other document content. The rest of the document content is available as a dict in the body property or as JSON as you are extracting with documentAsDataError() - both contain only the CDTDocumentRevision body i.e. not the _ prefixed metadata.
As @emlaver already showed the document _id of the CDTDocumentRevision is in the docId property and you can access it from there.
Only _ prefixed fields at the JSON document root are treated as CouchDB metadata, which is why you see a discrepency.

When you do a pull replication the documents from the server are inserted into the local data store.
When you query the local data store matching documents are returned as CDTDocumentRevision instances.
So the variable doc you are emumerating in result.enumerateObjects are instances of CDTDocumentRevision from the local datastore.
In a CDTDocumentRevision the root level _ prefixed document metadata is processed separately from other document content. The rest of the document content is available as a dict in the body property or as JSON as you are extracting with documentAsDataError() - both contain only the CDTDocumentRevision body i.e. not the _ prefixed metadata.
As @emlaver already showed the document _id of the CDTDocumentRevision is in the docId property and you can access it from there.
Only _ prefixed fields at the JSON document root are treated as CouchDB metadata, which is why you see a discrepency.

@ricellis As you explained, I did same thing and now I can see  "_id"
 doc.value(forKey: "_docId")
While parsing each document explicitly I have to add to each document

As shown in above screenshot, I should append to the document to parsed.
If this is the case, then suppose there are 100+ document it will be very cumbersome technically because already a parameter is coming and again I have to append to document.

@ricellis As you explained, I did same thing and now I can see  "_id"
 doc.value(forKey: "_docId")
While parsing each document explicitly I have to add to each document

As shown in above screenshot, I should append to the document to parsed.
If this is the case, then suppose there are 100+ document it will be very cumbersome technically because already a parameter is coming and again I have to append to document.

it will be very cumbersome technically because already a parameter is coming and again I have to append to document

I don't really understand this - if you use your own types instead of the provided CDTDocumentRevision interface then by necessity you have to convert between that interface and your own type. If you want to use the metadata as well as the "body" then you have to extract the metadata from the CDTDocumentRevision into your type too.
You're already looping the docs in the query results to convert them to your Swift model types, it's a 1-line addition to do the assignment of the CDTDocumentRevision's docId property to your type's own id property isn't it?
e.g.
        result.enumerateObjects { [weak self] (doc, idx, stop) -> Void in
            autoreleasepool {
                do {
                    let data: Data = try doc.documentAsDataError()
                    // Decode data to object
                    let jsonDecoder = JSONDecoder()
                    let document: T? = try jsonDecoder.decode(T.self, from: data)
                    if document != nil {
                        document.id = doc.docId //************ why not just add this 1 line? *************
                        documents.append(document!)
                    }
                } catch {
                    let error = NSError(domain: "CloudantBaseDao", code: 404, userInfo: [NSLocalizedDescriptionKey:
                        error.localizedDescription, "DB": self?.databaseName ?? "", "Device ID": AppGlobals.deviceID, "Query": query])
                    Crashlytics.crashlytics().record(error: error)
                    Log.error("CloudantBaseDao Error: \(error), Document ID: \(doc.docId ?? "No ID Found")")
                }
            }
        }

It appears anyway that the docId in the CDTDocumentRevision is correctly set so there is no bug with the pull replication or the local query result here that I can see.

I found this way of adding  "_id" back to document and it works for me.
doc.body.addEntries(from: ["id": doc.docId ?? ""])
Thanks for the help @ricellis @emlaver


refactor(logging): Moving to native python loggers

removing codypy/logger.py
deprecating is_debugging and debug_method_map
deprecating explicit sys.exit() calls in favour of exceptions
introducing custom exceptions for error handling
using logging.info & logging.debug instead of prints. the decision to which one to use is rather arbitrary for now.
using specific stream logger (messaging.stream) for logging in-progress messages with DEBUG level.
added a few TODOs

Some extra comments in relation to this MR:

library itself should never print(), that is the privilege of the application embedding the library (e.g. cli.py or main.py)
library should not configure the logger (same as above)
library should never call sys.exit() but raise exceptions instead

I did not test these changes due to lack of time. The other branch however has been tested well :)


Cannot import manim package from inside the PyCharm

I installed manim along with all dependencies required for MacOS (Macbook M2).
Current version of manim is v0.17.3.
When trying to install manim from pycharm, the following report is being received.
Collecting manim
Using cached manim-0.17.3-py3-none-any.whl (530 kB)
Collecting mapbox-earcut<2.0.0,>=1.0.0
Using cached mapbox_earcut-1.0.1-cp310-cp310-macosx_11_0_arm64.whl (71 kB)
Collecting numpy<2.0,>=1.19
Using cached numpy-1.24.3-cp310-cp310-macosx_11_0_arm64.whl (13.9 MB)
Collecting scipy<2.0.0,>=1.7.3
Using cached scipy-1.10.1-cp310-cp310-macosx_12_0_arm64.whl (28.8 MB)
Collecting moderngl<6.0.0,>=5.6.3
Using cached moderngl-5.8.2-cp310-cp310-macosx_10_9_universal2.whl (192 kB)
Collecting watchdog<3.0.0,>=2.1.6
Using cached watchdog-2.3.1-cp310-cp310-macosx_11_0_arm64.whl (90 kB)
Collecting colour<0.2.0,>=0.1.5
Using cached colour-0.1.5-py2.py3-none-any.whl (23 kB)
Collecting moderngl-window<3.0.0,>=2.3.0
Using cached moderngl_window-2.4.3-py3-none-any.whl (369 kB)
Collecting screeninfo<0.9,>=0.8
Using cached screeninfo-0.8.1-py3-none-any.whl (12 kB)
Collecting svgelements<2.0.0,>=1.8.0
Using cached svgelements-1.9.3-py2.py3-none-any.whl (135 kB)
Collecting manimpango<0.5.0,>=0.4.0.post0
Using cached ManimPango-0.4.3.tar.gz (4.1 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pydub<0.26.0,>=0.25.1
Using cached pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting skia-pathops<0.8.0,>=0.7.0
Using cached skia_pathops-0.7.4-cp310-cp310-macosx_10_9_universal2.whl (2.9 MB)
Collecting rich!=12.0.0,>=6.0
Using cached rich-13.3.4-py3-none-any.whl (238 kB)
Collecting decorator<6.0.0,>=5.0.7
Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting srt<4.0.0,>=3.5.0
Using cached srt-3.5.3-py3-none-any.whl
Collecting networkx<3.0,>=2.5
Using cached networkx-2.8.8-py3-none-any.whl (2.0 MB)
Collecting cloup<0.14.0,>=0.13.0
Using cached cloup-0.13.1-py2.py3-none-any.whl (51 kB)
Collecting click-default-group<2.0.0,>=1.2.2
Using cached click_default_group-1.2.2-py3-none-any.whl
Collecting click<=9.0,>=7.2
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting pycairo<2.0,>=1.21
Using cached pycairo-1.23.0.tar.gz (344 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting tqdm<5.0.0,>=4.62.3
Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting Pillow<10.0,>=9.1
Using cached Pillow-9.5.0-cp310-cp310-macosx_11_0_arm64.whl (3.1 MB)
Collecting Pygments<3.0.0,>=2.10.0
Using cached Pygments-2.15.1-py3-none-any.whl (1.1 MB)
Collecting requests<3.0.0,>=2.26.0
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Collecting isosurfaces==0.1.0
Using cached isosurfaces-0.1.0-py3-none-any.whl (10 kB)
Collecting glcontext<3,>=2.3.6
Using cached glcontext-2.3.7-cp310-cp310-macosx_10_9_universal2.whl (12 kB)
Collecting pyrr<1,>=0.10.3
Using cached pyrr-0.10.3-py3-none-any.whl (46 kB)
Collecting pyglet>=2.0dev23
Using cached pyglet-2.0.5-py3-none-any.whl (831 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Collecting charset-normalizer<4,>=2
Using cached charset_normalizer-3.1.0-cp310-cp310-macosx_11_0_arm64.whl (123 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting markdown-it-py<3.0.0,>=2.2.0
Using cached markdown_it_py-2.2.0-py3-none-any.whl (84 kB)
Collecting Cython
Using cached Cython-0.29.34-py2.py3-none-any.whl (988 kB)
Collecting pyobjc-framework-Cocoa
Using cached pyobjc_framework_Cocoa-9.1.1-cp310-cp310-macosx_10_9_universal2.whl (390 kB)
Collecting mdurl~=0.1
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting multipledispatch
Using cached multipledispatch-0.6.0-py3-none-any.whl (11 kB)
Collecting pyobjc-core>=9.1.1
Using cached pyobjc_core-9.1.1-cp310-cp310-macosx_10_9_universal2.whl (735 kB)
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: manimpango, pycairo
Building wheel for manimpango (pyproject.toml): started
Building wheel for manimpango (pyproject.toml): finished with status 'error'
Building wheel for pycairo (pyproject.toml): started
Building wheel for pycairo (pyproject.toml): finished with status 'error'
Failed to build manimpango pycairo
error: subprocess-exited-with-error
× Building wheel for manimpango (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [41 lines of output]
:98: RuntimeWarning: pkg-config doesn't exists or doesn't seem to work
We assume that you give the compiler flags using
corresponding environment variables.
:98: RuntimeWarning: pkg-config doesn't exists or doesn't seem to work
We assume that you give the compiler flags using
corresponding environment variables.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-universal2-cpython-310
creating build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/_version.py -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/init.py -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/utils.py -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/cmanimpango.pxd -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/cairo.pxd -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/register_font.pxd -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/pango.pxd -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/glib.pxd -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/enums.pyx -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/register_font.pyx -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
copying manimpango/cmanimpango.pyx -> build/lib.macosx-10.9-universal2-cpython-310/manimpango
running build_ext
building 'manimpango.cmanimpango' extension
creating build/temp.macosx-10.9-universal2-cpython-310
creating build/temp.macosx-10.9-universal2-cpython-310/private
creating build/temp.macosx-10.9-universal2-cpython-310/private/var
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3/manimpango_0765aebce6ea44dcb6174d8bbad05a1e
creating build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3/manimpango_0765aebce6ea44dcb6174d8bbad05a1e/manimpango
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g "-I/Users/davidarzumanyan/PycharmProjects/CS205 - Project3/venv/include" -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c /private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3/manimpango_0765aebce6ea44dcb6174d8bbad05a1e/manimpango/cmanimpango.c -o build/temp.macosx-10.9-universal2-cpython-310/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3/manimpango_0765aebce6ea44dcb6174d8bbad05a1e/manimpango/cmanimpango.o
/private/var/folders/jr/c2nt1b7s3yn6lfs95hm8g1cr0000gn/T/pip-install-z8khdxz3/manimpango_0765aebce6ea44dcb6174d8bbad05a1e/manimpango/cmanimpango.c:767:10: fatal error: 'cairo.h' file not found
#include "cairo.h"
^~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for manimpango
error: subprocess-exited-with-error
× Building wheel for pycairo (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [12 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-universal2-cpython-310
creating build/lib.macosx-10.9-universal2-cpython-310/cairo
copying cairo/init.py -> build/lib.macosx-10.9-universal2-cpython-310/cairo
copying cairo/init.pyi -> build/lib.macosx-10.9-universal2-cpython-310/cairo
copying cairo/py.typed -> build/lib.macosx-10.9-universal2-cpython-310/cairo
running build_ext
'pkg-config' not found.
Command ['pkg-config', '--print-errors', '--exists', 'cairo >= 1.15.10']
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pycairo
ERROR: Could not build wheels for manimpango, pycairo, which is required to install pyproject.toml-based projects
[notice] A new release of pip available: 22.3.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip
It is very weird that in the terminal no issues are found with manim. But for some reason I am unable to import it in the pycharm.
Also, I did run - source /Users/userName/PycharmProjects/CS205 - Project3/venv/bin/activate
pip install manim. Terminal didn't give any issues on this.

Thank you for an effort to help, the problem was solved. I decided to try to install the pkg-config again, and when installing the terminal showed that it was already installed and up to date. After which I decided to try to install the package one more time from pycharm and this time the package was successfully installed.


Kosmos 2 - training hangs when no_prefetch=False

Describe the bug
Model I am using (UniLM, MiniLM, LayoutLM ...): Kosmos 2
The problem arises when using:

the official example scripts: I try to run the simplest finetuning accordingly to the instructions.

A clear and concise description of what the bug is:
The training script hangs with:
2023-09-10 16:12:56 | WARNING | infinibatch.iterators | trying to fetch item, but prefetch buffer is empty

When I can the dataloader with no_prefetch=True the script works fine. Is it possible that this happens because there are not enough tsv files? what may cause the script to hand when no_prefetch=False?
To Reproduce
Steps to reproduce the behavior:

Create a single tsv file and run the given cmd for training.

Hi @yuvalkirstain, do you run the job in the debug mode? no_prefetch=False can improve the speed of loading data but can not used in the debug mode. The warning is fine and it should only appear at the beginning of the training.

Thank you for the help.

What is the minimum number of tsv files that are recommended to use for debugging and training?
How much time are we expected to wait when we see the infinibatch.iterators | trying to fetch item, but prefetch buffer is empty warning?

Also, it can be helpful for you to release sample data files (that are much smaller than the ones used to originally train the models) that will enable users to quickly run the code out-of-the-box and understand how the data should look like, and the structure of data. Currently users have to understand this from the code itself.


task: GSheets Scope Label Change

Description

The following PR changes the label values seen by the user in the scope selection drop-down menu while creating a Google Sheet data source.
The changes accommodate for one specific scope that was present in the previous implementation and is not present now i.e read/write without delete permission for which a new scope has been added.
Also with these changes the migration for old data sources is not required since we have added all the previously used scopes here.
Figma Link: https://www.figma.com/file/TcFhqEbAc8ymHTRF5wR1qv/Limited-GSheet-Access?node-id=506-32761
Notion link for migration: https://www.notion.so/appsmith/Google-Sheet-Migration-ab7bbfbdae1f40189a957b6ae5374145

Fixes #22327
Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

Manual

Checklist:
Dev activity

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test

/ok-to-test

/build-deploy-preview

@PiyushPushkar02 @sneha122
Issue on PR

In cancel the scope is not displayed to the user on review page(https://www.loom.com/share/67ba9419b74a421bbff15d466fba8357)
We are telling the user read/write and taking permission to delete from google. However on the review screen after authorise we show them its read create and edit but delete action also works (https://www.loom.com/share/7862d2b7ff424c7b85f1a5dad8acf3cb)

Hi, @prapullac  for the 2nd issue mentioned above the delete permission has a different meaning. For " Read/ Write/ Delete " the delete permission says a user can delete an entire sheet while for " Read/ Write" it means a user can delete a row inside a sheet but not the entire sheet.


Drop Swift 5.5 & 5.6 support

Describe the feature
Remove support for Swift 5.5 & 5.6 on aws-sdk-swift and smithy-swift, leaving Swift 5.7 as the minimum supported Swift version.  For reference, Swift 5.7 was introduced in Xcode 14.0 in September 2022.
Specific tasks:

Set swift-tools-version and other minimum Swift version limits in our targets from 5.5 to 5.7.
Remove conditional code for Swift version 5.5 & 5.6.
Remove Swift 5.5 & 5.6 from CI tasks, replace with versions up to latest Swift where needed.
Remove Catapult builds using Swift 5.5 & 5.6

Use Case

To allow for the use of more modern Swift
To reduce the burden of supporting old Swift versions
To reduce the complexity and build time on build & CI systems

Proposed Solution
Specific tasks:

Set swift-tools-version and other minimum Swift version limits in our targets from 5.5 to 5.7.
Remove conditional code for Swift version 5.5 & 5.6.
Remove Swift 5.5 & 5.6 from CI tasks, replace with versions up to latest Swift where needed.
Remove Catapult builds using Swift 5.5 & 5.6, replace with versions up to latest Swift where needed.

Other Information
No response
Acknowledgements

[ ] I may be able to implement this feature request
[ ] This feature might incur a breaking change

This task is blocked by amplify-swift currently supporting Swift 5.6.
This task may be unblocked once amplify-swift raises their minimum Swift version to 5.7.

Be sure to inform AWS doc writers prior to the release of this change in requirements.

Amplify dropped Swift 5.6 support in amplify-swift 2.18.0 so we are now free to move to Swift 5.7 as our minimum for SDK and smithy-swift.
Removing the "Blocked" label accordingly.

The CI part of this ticket will be accomplished in #1132 and #1133.

@shepazon Tagged you here for your info

Build system has been updated to run Xcode 14.0 / Swift 5.7 minimum, Xcode 15.0 / Swift 5.9 max.
Internal ref: CR-102886049

Released in 0.28


Catch JS errors in student profile PDF so that wkhtmltopdf always completes

The idea here is that JS errors like in https://github.com/studentinsights/studentinsights/issues/1276#issuecomment-347378944 shouldn't break the whole PDF rendering, but be caught so that the "ready" value can be set regardless and rendering can finish.

Talking with @gabivoicu about having this be her next feature in the queue!
Talking out loud with @gabivoicu, she's suggesting that our goal for this should be to (1) surface errors for developers that occur in the student profile pdf JS and (2) add an error to the UI for educators when their PDF can't render. I think that's a really strong suggestion!

Describing the scenario for this issue:
  window.shared.StudentProfilePdf = {
    load: function() {

      throw();    // An unexpected bug has creeped into the codebase!

      const attendanceData = $('#serialized-data').data('attendance-data');
      const filterDateRange = $('#serialized-data').data('graph-date-range');
...

When the code above  throws, as developers we want to see an error in our console instead of having the PDF silently hang forever and ever.

Removing the help wanted tag because @gabivoicu is on this one!

@alexsoble Update on my progress after working on this today: none 😔
I have tried all of the things we talked about in the call yesterday (to the best of my understanding at least), and have not been able to get the errors logged or the user notified.
One thing that I have been able to do is verify that PDFs generated while an error occurred are identical to ones where an error did not occur (at least in the couple I tested). I could put that in a PR, if you think that makes sense.
I feel I've reached the limits of my JS understanding, so it might make sense for someone else to pick it up from here.
Let me know if that makes sense or if you think I should work on this a bit more. (3.25 hours total so far, fyi).

Hey there @gabivoicu! Thanks for the update. I agree it might make sense to shift to another issue. You've gained a lot of context and learning around the Student Report PDF code, so let's put that to use in another Student Report issue.
I think #1257 could be a good next item to work on. It involves strengthening testing for the Student Report. Let me add some more comments and background there, we can discuss more in the comments section of #1257.

@alexsoble Sounds good, I will check that one out!


test: aggregate deveice r/w bps/iops cases to reduce time

Signed-off-by: Allen Sun<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
I found that in the cli integration test of travis CI, PouchRunDeviceSuite test suite would cost lots of time. Then I try to aggregate some similar test cases into a single one to see if we could make to save time in test.
Four cases takes about 1min, and we need to check if the aggregation would reduce 3/4 of it.
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:59: PouchRunDeviceSuite.TestRunDevicePermissions	3.987s
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:140: PouchRunDeviceSuite.TestRunDeviceReadBps	14.241s
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:220: PouchRunDeviceSuite.TestRunDeviceReadIops	14.296s
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:123: PouchRunDeviceSuite.TestRunDeviceWithBadDevice	0.918s
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:180: PouchRunDeviceSuite.TestRunDeviceWriteBps	14.303s
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:261: PouchRunDeviceSuite.TestRunDeviceWriteIops	14.330s

In addition, in the test case, I try to use the same format and use fmt.Sprintf() to combine strings.
Ⅱ. Does this pull request fix one issue?

this is related to https://github.com/alibaba/pouch/issues/1184.
Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
just refactor test cases, no need to add case.
Ⅳ. Describe how to verify it
pass the CI
Ⅴ. Special notes for reviews
none

After this pull request, the test case PouchRunDeviceSuite.TestRunDeviceReadWriteBpsIops only takes about 14s reducing 42s.
PASS: /home/travis/gopath/src/github.com/alibaba/pouch/test/cli_run_device_test.go:157: PouchRunDeviceSuite.TestRunDeviceReadWriteBpsIops	14.302s

LGTM

Thanks for your review. @chuanchang I will merge this.


Implement a gym map?

Would it make sense to implement a map of gyms to be generated from frontmatter data?
Or Google Maps is good enough?
This is obviously not urgent, let's see what other points of interest could be mapped in the same way before writing code and maintaining data set for a very niche feature.

Relevant plugin: https://github.com/altrdev/hugo-leaflet

Leaflet plugin requires font awesome. Does this plugin satisfy the dependency? https://github.com/hugo-mods/icons#icons

Storing various service geolocations in front-matter could also be useful to implement data export to other mapping apps like my travel planner i've been working on

There's a fork of leaflet plugin that fixes (?) some bug


Caching not working by default

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (7545c7a 2024-03-12T00:16:54.294788023Z)
What command(s) is the bug in?
No response
Operating System
Linux
Describe the bug
When running forge build twice in a row, with no changes, the previous build isn't cached.  This affects other commands as well but is most apparent with build.
with an older release forge 0.2.0 (9f6bb3b 2024-03-11T00:16:01.716842530Z), notice that the cache is recognized and build is not repeated.
forge build
[⠃] Compiling...
No files changed, compilation skipped

but with forge 0.2.0 (7545c7a 2024-03-12T00:16:54.294788023Z) (the buggy version) the build cache is not recognized
forge build
[⠆] Compiling...
[⠰] Compiling 1 files with 0.5.17
[⠔] Compiling 324 files with 0.8.15
[⠑] Compiling 129 files with 0.8.19
[⠊] Solc 0.5.17 finished in 38.53ms
....

note that between running these two commands, I didn't modify any source files.

cc @klkvr, related to https://github.com/foundry-rs/foundry/pull/7358 maybe?

hey @onbjerg , I am realizing that I am requesting build_info in foundry.toml, so I guess this is techncially working as expected, but it threw me off 🤔
would it be appropriate to log a message that caching is disabled if build_info is enabled?

yeah, it could be, @ClaytonNorthey92 do you have build_info = true in your foundry.toml?

yeah, it could be, @ClaytonNorthey92 do you have build_info = true in your foundry.toml?

@klkvr yes I am just realizing that, mentioned here with a question https://github.com/foundry-rs/foundry/issues/7379#issuecomment-1992009683

Yeah, I think such warning could make sense at least when build_info = true is appearing in foundry.toml

@klkvr also, how do you feel about changing the --build-info flag to a boolean, so you could:
forge build --build-info=0

and that would disable build_info, thus enabling caching
the reason that I ask is that I am running forge on a code base that I don't own, so I don't want to modify their foundry.toml, but we also don't use the build info files, so it would be convenient to override that via the cli.  what do you think?

Just want to point out that #7358 causes a significant slowdown in repos that set build_info=true in their foundry.toml. A full recompilation occurs every time the user runs forge test or forge script. This recompilation can take minutes if the project has a lot of contracts or uses the IR compilation pipeline. I don’t think Hardhat has this issue because they recompile only the necessary contracts and their parent contracts, which solves the original issue without force recompiling

Just want to point out that #7358 causes a significant slowdown in repos that set build_info=true in their foundry.toml. A full recompilation occurs every time the user runs forge test or forge script. This recompilation can take minutes if the project has a lot of contracts or uses the IR compilation pipeline. I don’t think Hardhat has this issue because they recompile only the necessary contracts and their parent contracts, which solves the original issue without force recompiling
@klkvr

@sam-goldman foundry would not recompile any of the files if them or their imports have not been changed. Thus, unchanged parent contract would not be recompiled. The #7358 was created with idea that expected behavior is to have all sources in build-info which is only possible when no cache is enabled.
I feel like we might want to use similar approach as hardhat here: hardhat stores build-info by default on each compiler run and keeps a mapping from contract artifact to build-info file, removing build-info file once no contract are pointing at it. That way, in the case from original issue, latest build info would still only include child contract, but it would be possible to match parent contract to build-info with data for it.
This would be a bit hardher to parse, but will be more precise and would not reauire disabling cache. Do you think such UX would make sense @sam-goldman?
CC @mattsse

I started having issues related to cache as I needed both build info and cache files when #7358 hit; it silently disabled cache when build info was set to true (e.g.: it's a requirement for OZ Upgrades Foundry plugin)
Cache files - Seems to be needed to run a simple script otherwise, it throws
%> forge script SCRIPTNAME --broadcast --rpc-url http://localhost:8545 --unlocked --sender 0xDEADBEEF
[⠊] Compiling...
[⠊] Compiling 105 files with 0.8.20
[⠃] Solc 0.8.20 finished in 26.92s
Compiler run successful!
Error:
Could not open compiler cache

Context:
- "/home/magicking/PATH_TO/cache/solidity-files-cache.json": No such file or directory (os error 2)

And if the build info is disabled, the OZ Upgrades can't do all the validations.
Quite annoying, activating more build information shall not deactivate the cache; seems very counterintuitive!


Mission control file store form values get persisted across all the supported storage systems

Describe the bug
If I provide any value for directory path field in local store, this same value gets persisted for amazon region field.
same for the bucket name of aws get persisted for bucket name in gcp.

Expected behaviour
Actual behaviour
Steps to reproduce
Check out this video https://app.clickup.com/t/199b2r?comment=15965827
Your environment

Space Cloud version:
OS:
Kubernetes or Docker:
The client api version (if applicable):
Browser (if applicable):

If this bug restricts your use of space-cloud, give it a thumbs up reaction, so that we can determine which bugs need to be fixed immediately.
👍

Check out this link for more information Video

Done. Coming in v0.19.3.


Add ability to set dev compose file which will be used additionally

For example, you can now do this in your makefile
export COMPOSE_DEV_FILE?=docker/docker-compose.dev.yaml

-include $(shell curl -sSL -o .build-harness "https://raw.githubusercontent.com/mintel/build-harness/master/templates/Makefile.build-harness"; echo .build-harness)

without setting COMPOSE_PROJECT_NAME in your makefile, the services will now be named as so:
Creating svc_1966417_501_gateway-cache_1 ... done
Creating svc_1966417_501_gateway-db_1    ... done
Creating svc_1966417_501_gateway_1       ... done
svc_1966417_501_gateway-cache_1 is up-to-date
svc_1966417_501_gateway-db_1 is up-to-date
svc_1966417_501_gateway_1 is up-to-date


Dropdown Auto Close Not Working When Wrapped In Another Component

@JamieTran is seeing this in Consistent Eval with the labs filter and in FACE with a normal dropdown.  They both use primary-secondary template but he was able to reproduce this outside of that, and I created a simple reproduction inside the dropdown demo: https://github.com/BrightspaceUI/core/compare/demonstrate_broken_auto_close

It seems like https://github.com/BrightspaceUI/core/blob/master/components/dropdown/dropdown-content-mixin.js#L354 isn't triggered when expected.

I'm gonna delete the branch above, which should be fine because:

This should be fixed 🎉
@margaree added a visual-diff test to catch if this breaks again


Feedback for “CldUploadWidget Examples”

Here's a clear and concise email draft for contacting the Next.js team regarding the issue:

Subject: Issue with CldUploadWidget Handling of Image Upload and Folder Structure
Message:
Hi Next.js Team,
I'm encountering an issue with CldUploadWidget where not all selected files seem to appear in the Cloudinary response as expected. Here’s a summary of the problem:

When I upload multiple images (for example, 4), only 3 URLs are returned from result.info.files in handleUploadSuccess. However, when I check Cloudinary, I see that all 4 images have been uploaded.
Additionally, although I've set options.folder to title (to create a folder with the product title name), the files are not always placed within this specified folder. It seems like only 3 files end up in the folder, and one is missing from the designated location, even though all have uploaded.

Here is the code setup for the CldUploadWidget:
<CldUploadWidget
  uploadPreset="tqw4r7p7"
  onUpload={handleUploadSuccess}
  options={{
    resourceType: 'auto',
    multiple: true,
    folder: title,
  }}
>
  {({ open }) => (
    <button
      onClick={() => {
        if (!title) {
          alert("Please enter a title before uploading.");
        } else {
          open();
        }
      }}
      className="flex min-w-[84px] max-w-[480px] cursor-pointer items-center justify-center overflow-hidden rounded-xl h-8 px-4 bg-green-600 text-white text-sm font-medium leading-normal"
    >
      Upload (Max: 5)
    </button>
  )}
</CldUploadWidget>

Could you help me understand why only a subset of URLs is returned, and why not all files are stored in the designated folder? Is there a workaround or a setting adjustment needed?
Thanks in advance for your assistance!
Best regards,
Syed Mutahir

Please use onSuccess instead of onUpload. onUpload is deprecated as noted in the documentation. https://next.cloudinary.dev/clduploadwidget/configuration#callback-functions


ion-echo example, add TURN settings and RTT on top of remote video

These changes will make it possible for users to perform the echo testing while using a TURN service between them and the server.
For the operation to work I had to add an additional phase, "Connect", to be performed before the "Join".

They also add an indication of the RTT, on top of the remote video, near the bitrate.

LGTM


'DataFrame' object has no attribute 'map'

bug due to pandas DataFrame`s update
map is not a function for DataFrame
fixed in version 0.0.8


Inconsistency in hover color in buttons

Prerequisites

[X] I have searched for duplicate or closed issues
[X] I have validated any HTML to avoid common problems
[X] I have read the contributing guidelines

Describe the issue
Not really a bug but inconsistent behavior not sure if is intended to be this way.
On hover in buttons Primary, Secondary, Success and Danger the background becomes darker in the other hand on buttons Warning and Info the background becomes lighter and there is no change on Dark and Light buttons.
Reduced test cases
https://getbootstrap.com/docs/5.2/components/buttons/#examples
What operating system(s) are you seeing the problem on?
Windows
What browser(s) are you seeing the problem on?
Chrome
What version of Bootstrap are you using?
v5.2.0-beta1

The button variants are handled with a mixin based on the contrast of our background and foreground colors. We darken or lighten based on if the contrast would be lessened on hover/active/etc. It's not always a guarantee that the hover and active states will consistently darken or lighten, which ultimately gives the system some more flexibility.
That said, we did ship some tweaks to buttons in v5.2.0 stable today, so hopefully that helps you out a bit with the dark and light buttons in particular.


DownloadPipelineArtifact task patterns not working

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: Download Pipeline Artifacts task
list here (V# not needed):
https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/DownloadPipelineArtifact (V2)
Environment

Server - Azure Pipelines or TFS on-premises?

Server - Azure Pipelines, acc:schwarzit, project:kaufland.bads-ai, repo:KAP.airflow

Agent - Hosted or Private:

Private, RHEL 7.7 (Maipo), vsts-agent-linux-x64-2.174.1
Issue Description
Expect matching pattern to match file correctly. Minimatch pattern matches 0 files.
Start downloading artifact - airflow.cfg_sometext-scratched_chd
Minimatch patterns: [*_chd]
Filtered 0 files from the Minimatch filters supplied.
Downloaded 0.0 MB out of 0.0 MB (100%).
Patterns tested:
*chd*
*chd
*_chd
All return 0 matches.

Possible root cause is the '.' in the filename
https://github.com/isaacs/minimatch/issues/82
To be tested


Add versioning to JSON table configuration files.

Add when we need to have versioning, not needed for 1.0


Weird crash on FBLoginManager.Events only when not in debugging mode

Hey there!
I am facing a weird issue accoriding this module. When i try to run my app with FBLogin module it crashes with this error message:

Now the weird thing is, that when i am in debug mode, than this works great....Any ideas how to solve this ?!

see if issue #110 helps

Great! Worked fine, thanks @lwhiteley


Migrat to tikv-jemallocator

I believe that is recommended to use tikv-jemallocator instead of what is currently being used jemallocator. I believe the crates are identical however it is recommended to use tikv-jemallocator for new projects:

The project is also published as jemallocator for historical reasons. The two crates are the same except names. For new projects, it's recommended to use tikv-xxx versions instead.

This becomes relevant when integrating with other projects (such as reth) which use tikv-jemallocator.

Also relates to #205.


EMOJI_REGEX throws Invalid regular expression flags on node v18

Hello, the following line breaks projects running on node 18
export const EMOJI_REGEX = /^\p{RGI_Emoji}+$/v;

Node js 18 does not support regex /v flag so node throws an error
SyntaxError: Invalid regular expression flags

Easy fix:

export const EMOJI_REGEX = (() => {
  try {
    return new RegExp('^\\p{RGI_Emoji}+$', 'v');
  } catch {
    return /^[\p{Extended_Pictographic}\p{Emoji_Component}]+$/u;
  }
})();

The bigger question is whether we should stick with the current code and force people to upgrade to Node.js v20. I don't plan to implement a try/catch workaround, but I can consider not using this feature until v2 of Valibot to give devs more time.

Thanks for your research! Since this regex is not too important, I think I will follow your recommendation.


Admin Dashboard Reports: Platform Tips section

Part of https://github.com/opencollective/opencollective/issues/4300
Design file: Figma

Front-end

[ ] Use HostMetrics to get the platform tips collected and the platform tips owed amounts
[ ] Add PeriodFilter to make sure we can filter by the date

API

[ ] Add API Query to get historical data; Check with legacy data (pre-ledger migration)

Add API Query to get historical data;

@SudharakaP I'm not sure what you mean by that, do we really need that since there's no historical chart in there?

Add API Query to get historical data;

@SudharakaP I'm not sure what you mean by that, do we really need that since there's no historical chart in there?

Yes looking back, I don't think we need this since there's no historical chart. I based this off of the document and I think we put it there thinking there's a chart; https://docs.google.com/document/d/1RwC8agICM9CL0IBM9-dtpO2fGPxGKSy5z4hUDdyV-B8/edit. 🤔


fix listener access log test

Fixes https://github.com/istio/istio/issues/39619

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@howardjohn @hzxuzhonghu PTAL


fix: dep versions

Closes #
Describe your proposed changes here.

[ ] CHANGELOG.md updated with a link to the PR (not the Issue)
[ ] Well-formatted commit messages
[ ] Rebased/mergeable
[ ] Tests pass
[ ] http/swagger.yml updated (if modified Go structs or API)
[ ] Documentation updated or issue created (provide link to issue/pr)
[ ] Signed CLA (if not already signed)

Closing as outdated (trying to prune our old PR queue)


Attributes are not sorted when performing toJSONString

Is there any reason why the attributes of an object N are not sorted by the order of the mapping in the result String after performing toJSONString(object: N, prettyPrint: Bool) ? What is the sort criteria for the JSON fields?

JSON String generation is handled by NSJSONSerialization.dataWithJSONObject(). ObjectMapper simply passes it a dictionary of the mapped object. So the sorting of the generated JSON string is out of our control

Isn't really a way around this? The GSON library for Android allows to add custom JsonSerializers to define the order of the elements. There's got to be a way.

You have to implement OrderedDictionary to achieve ordered keys access. Not sure this feature will be so useful. In addition JSON is unordered by spec

Do you have any example on how to do this? I'm not being able to find anything. Thanks.

Maybe you get me wrong: OrderedDictionary should be used in ObjectMapper implementation. It's relativitely easy to implement, i guess(users will receive possibility to return list of keys with proper order).
You can check out this SO question which contains several links to OrderedDictionary implementation. If you have time for experiments: please check if it fix JSON serializing order and maybe ObjectMapper can receive new feature(with @tristanhimmelman approvement obviously)

Do you mean using this class to convert to JSON?
http://www.cocoawithlove.com/2008/12/ordereddictionary-subclassing-cocoa.html
As pointed in the SO question you've mentioned, this feature of maintaining the order of the attributes is really important "if you serialize the json to NSData and you use an MD5 string to determine whether the data has changed, it's good to know that it hasn't."


VS Code sluggish typing & interaction experience

Type: Performance Issue
When I type my code to the editor the response time delays about 500-600 ms plus when I used VS Code in my Ubuntu 22 (Jammy Jellfish) Linux in my Dell Chromebook 11 it became sluggish all the time even though when I saw the usage of VS Code in task manager the memory usage is less.
VS Code version: Code 1.82.2 (abd2f3db4bdb28f9e95536dfa84d8479f1eb312d, 2023-09-14T05:51:20.981Z)
OS version: Linux x64 4.4.275-20726-g204ae1828356
Modes:

System Info

Item
Value

CPUs
Intel(R) Celeron(R) CPU  N2840  @ 2.16GHz (2 x 2582)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: disabled_offopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: disabled_softwarevulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
2, 1, 1

Memory (System)
3.77GB (2.01GB free)

Process Argv
--unity-launch

Screen Reader
no

VM
0%

DESKTOP_SESSION
xfce

XDG_CURRENT_DESKTOP
XFCE

XDG_SESSION_DESKTOP
undefined

XDG_SESSION_TYPE
undefined

Process Info
CPU %	Mem MB	   PID	Process
    1	   143	  6212	code main
    0	    43	  6217	   zygote
    1	    81	  6276	     gpu-process
    0	     0	  6218	   zygote
    0	    43	  6224	     zygote
    0	     8	  6227	       zygote
    0	   216	  6308	window [1] (● +page.svelte - lgub-hrms - Visual Studio Code)
    3	    93	  6819	window [2] (Issue Reporter)
    0	    58	  6286	   utility-network-service
    0	   120	  6412	extensionHost [1]
    0	   205	  6466	     electron-nodejs (/usr/share/code/code --ms-enable-electron-run-as-node --experimental-modules /home/yerenzter/.vscode/extensions/svelte.svelte-vscode-107.11.0/node_modules/svelte-language-server/bin/server.js --node-ipc --clientProcessId=6412)
    0	    81	  6423	shared-process
    0	     0	  6833	     /bin/sh -c /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=
    0	     0	  6834	       /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=
    0	    73	  6424	fileWatcher [1]

Workspace Info
|  Window (● +page.svelte - lgub-hrms - Visual Studio Code)
|    Folder (lgub-hrms): 28 files
|      File types: js(6) json(2) svelte(2) eslintignore(1) cjs(1) gitignore(1)
|                  npmrc(1) prettierignore(1) prettierrc(1) md(1)
|      Conf files: jsconfig.json(1) package.json(1);

Extensions (2)

Extension
Author (truncated)
Version

materialize-snippets
len
1.4.5

svelte-vscode
sve
107.11.0

I'm experiencing similar issues. Typing fast drops some key strokes. I think it started in the last week or so (so after the August 2023 release). I'm on Debian 10 (buster) Dell XPS i7.

Please collect some profile information using these instructions: https://github.com/microsoft/vscode/wiki/Performance-Issues#sluggish-ui

Please collect some profile information using these instructions: https://github.com/microsoft/vscode/wiki/Performance-Issues#sluggish-ui

I tried this but the Trace-*.json files had 0 bytes. Tried a couple things; export from google-chrome works as expected, tried to search for solutions, found no help. Obviously a different issue, but didn't want to leave you hanging.

Closing as we have not heard from the reporter


Migrate to Project Pythia

@aladinor - is there interest in migrating to this Project Pythia and making this a formal cookbook?

Hi, @mgrover1. I would definitely like to migrate this into a formal cookbook. We need to make arrangements for authenticating and accessing data from the LASSO simulation.
Please let me know your thoughts.

Hey @aladinor - following up here! Most of the simulation data has been added to the pythia bucket. Would you be willing to transfer this repo over to Pythia and we can work through the rest of the steps to getting this published on the gallery?

Hi @mgrover1,
I’ve already transferred the Cacti repo to Pythia. Please let me know how I can assist/help in improving it and moving forward with publishing it as a Pythia cookbook.


Provide image options for UBC EOAS

Fixes https://github.com/2i2c-org/infrastructure/issues/3012
Removes the old stale default image from UBC as well: https://github.com/2i2c-org/infrastructure/issues/2674
Ref https://2i2c.freshdesk.com/a/tickets/905

Hmmm the PR looks fine, doesn't touch cluster cloudbank's hub lamission, but the automation sais it needs to be redeployed because files were changed. This must relate to https://github.com/2i2c-org/infrastructure/pull/3023 being merged after this PR opened, and before the comment by the automation were made.
I figure we can merge it no matter what, but feel free to rebase it first if you want.

Cloud Provider
Cluster Name
Hub Name
Reason for Redeploy

gcp
cloudbank
lamission
Following helm chart values files were modified: enc-lamission.secret.values.yaml, lamission.values.yaml


Some renames for KeyVault mgmt sdk

API change check for Azure.ResourceManager.KeyVault
API changes have been detected in Azure.ResourceManager.KeyVault. You can review API changes here


How can i change playlist music to redirect to another page using Link or onClick event

I am using this music player now i have to add 2 new features in it add (like) feature and to make it redirect when user plays on any song in the media players playlist. Any help will be appreciated. thanks

use onAudioPlayTrackChange or onAudioListsChange


Emphasize txt to json conversion while creating pull_secret.json

Wasted a few hours trying to figureout why kcli env  script hung after creating the VMs, downloading openshift client had failed due to my pull_secret.json content txt vs json, duh moment.

/ok-to-test

/approve
/lgtm


Seemingly webhook isn't sending along the hostname properly

Event though hostname was set to a proper URL it still appears to send localhost.

Great opportunity to test Gaia-bot to build an image and deploy to the testing environment.

Never mind. It's because the options are - and not --. 🤦


SAK-32679 - Adding an LTI Assignment type

Looks like the merge conflict worked itself out.

Lets get a rebase on this branch so that were not looking at commits that are from your merge. Also a lot of conflicts need to be resolved :D

This is kind of messed up so I am going to close this PR and re-pr it


how to understand device is rooted?

I want to know if the device has been rooted, or not? I looked at react native library as well as the expo and did not find anything.
Also, I want to have a list of installed apps on the phone.

tnx for the answer . try it and I'll say the result.

about react native rooted i cannot ejected my project. While this package is responding to the eject project.


can you shared C++ TensorRT inference version？

can you shared C++ TensorRT inference version？

hi, i am not familiar with tensorrt but shouldn't using something like torch2rt work?

hi, i am not familiar with tensorrt but shouldn't using something like torch2rt work?你好，我不熟悉tensorrt，但不应该使用像torch2rt工作的东西？

Mainly post-processing operations I do not know how to use cuda in C++ to do processing

it seems to have Resize and Convolution so it should be possible.

ok! thank you


Added my name

Hello @IKishan
Can you please do these steps to be able to merge your changes
open git bash
then do these steps( if you need help in any step ping me back)
cd git-learning
git checkout -b newbranch2
go to folder (git-learning) and create a new file yourname.html and add your name there
now come back to git shell
git status to check the status
git add .
git commit -m "new file added"
git push origin newbranch2
then enter your username and password
come back to this website and do comare and pull request.
Cheers,
Shahbaz


🚀 Feature: Add Subscriptions with Stripe for Swift

Appwrite Functions allow you to extend and customize your Appwrite server functionality by executing your custom code. 🤩 You can learn more at our official Appwrite Functions docs.
Your task is to implement the Subscriptions with Stripe function in Swift. You can look at the existing Appwrite Functions Templates in another coding language.
Your function should behave and be implemented similarly to existing Subscriptions with Stripe Node.js template.
Tasks summary:

[ ] Fork & clone Appwrite's templates repository
[ ] Create a branch feat-implement-subscriptions-with-stripe-swift
[ ] Write code for the function in folder swift/subscriptions-with-stripe/.
[ ] Test the function using Appwrite, provide a video.
[ ] Write a README.md explaining what the function does, how to use it, and customize it. Here is an example.
[ ] Submit a pull request to our templates repository

If you need any help, contact us on our Discord server.
Are you ready to work on this issue? 🤔 Let us know, and we will assign it to you 😊
Happy Appwriting!

Can you assign me this issue please?

Hi @Vinz000 have assigned the issue to you! Thank you for showing interest in contributing to Appwrite! Happy Hacktoberfest 🎃
Notes:
Please update us with your progress every 3 days, so that we know that you are working on it.
Join us on Discord - https://appwrite.io/discord to chat about Hacktoberfest and Appwrite!

@gewenyu99 I need further clarification, I noticed the implementation of this template using node.js calls the Stripe api. However, I can't call the Stripe api directly using swift so I'm unsure how to achieve the same functionality?

@gewenyu99 I see. Will work on this now!

Still working

Hey @Haimantika Can you assign this issue to me?

Hi @Vinz000 are you still working on it? If you have any updates, let us know.

@Haimantika I'm no longer working on it

Hey @Haimantika Can you assign this issue to me?

Hi @ShamanthOP the issue is yours now. Remember to keep us updated every 3 days.

Hey @Haimantika I won't be able to complete this due to some personal constraints.

Hello @Haimantika, Can I take this up?

Hi @Haimantika . I've worked with Stripe API and would love  to work on this issue. Can you please assign this to me ?

Closing issue due to inactivity and hacktoberfest being over, thank you for taking the time to be part of hacktoberfest this year :)


Documentation for Angular2 wanted

It is unclear to me how I am meant to import the superlogin-client in an angular2 project. Some more documentation would be nice.

This library is not meant to be for AngularJS but more for Vanilla.
If someone would be kind enough to write a documentation for AngularJS I have nothing against adding it though.

There is another library designed to integrate angular with Superlogin:
https://github.com/colinskow/ng-superlogin

The definition file is available in this repo (https://github.com/micky2be/superlogin-client/blob/master/superlogin-client.d.ts)
Feel free to use.
Then a simple import superlogin from 'superlogin-client';
I'm not sure what I could do more within this library for documentation.
For other details you can always check Angular or Typescript documentation.


Download SD videos

Would be possible to download the contents of the SD card ? I'd like to have a script that automatically backups all cameras videos, thanks in advance.

Hello, I too would love this feature.
Someone will need to discover a way how to get those videos from the camera and then I will add this to this lib.

Ok here we go, huge amount of work went into this and we need more help.
Thanks to a huge help from @Depau who RE the camera and figured how the camera loads recording and also wrote a lot of code, we were able to get to an almost-there state.
Almost is an important word here.
So here is a summary of whats going on:

Camera uses its own http/websockets like protocol on a custom port. Because of this we had to develop a completely new low-level code and not use something like requests (reimplement the digest, the whole communication with camera through sockets etc.). Depau documented how exactly it works here.
Camera 2-way encrypts all the streams with your cloud password so the communication is secure
Camera sends data in N chunks, and needs acknowledgment after every N chunks to send N more.
When one recording ends, another one starts automatically. Camera doesn't inform about this.
When all the recordings for the day are sent in the stream, camera responds with {'type': 'notification', 'params': {'event_type': 'stream_status', 'status': 'finished'}}

With the current code, we are able to get all the raw & decoded data for the whole day (currently its temporarily limited to 2000 sequences, gives around a minute or 2, for testing).
However, when we convert the TS data into mp4, we get a video having around 1.3FPS which is terrible.
Here is where we need help. There are 2 possibilities what is wrong here:
1.) We are missing something in the code. Currently we are getting all the binary data and decoding & connecting it together. It decodes fine and is also able to be converted, so I think this is not the case.
2.) The conversion which we are doing through ffmpeg (not part of the code) is wrong. Maybe we need to specify another input codec or something else. Converted video also doesn't have a sound. I think this is the culprit.
After we resolve the low fps issue we can continue with the code and:

Implement converting via the library directly, along with some nice functions
We could also split the video file according to data we have (start and end times of all the recordings) so that we get individual recordings for the whole day

If you have the python / ffmpeg experience and wish to help:
Download latest dev branch.
Run file attached to this comment. The file should use the downloaded library, not your installed one.
It will download a part of the day recordings and save it to ./output/ directory.
Try to get a smooth video file from it... and please let us know if you have any success!

Camera 2-way encrypts all the streams with your cloud password so the communication is secure

FYI people: An attacker who has a dump of the communication can still decrypt the video. Beware ;)

Anyway I sent a TS video sample to a friend of mine who's into video codecs, he's gonna check it out.

@Depau would you be able to share it, I'd like to try it to if possible. I'm away from my cameras until next year, so it would be awesome if you can share that sample.

@Depau if its mine sample please do not share publicly

Yeah I only have your sample, I'm not sending it around. I only showed it to my friend (i know him personally) and he said that to him it looks like the video is actually recorded at 1/2 fps, I have no idea why that would be.
Maybe someone else can try to retrieve some samples and publish them so people can try to figure it out.

I'm going to try when I'm back to my house Jan 20th.

Same for me.
Playing the .ts file on kmplayer gives me a video with frames every 2 seconds + no audio.
Same result after converting it to MP4.

Same for me.
Playing the .ts file on kmplayer gives me a video with frames every 2 seconds + no audio.
Same result after converting it to MP4.

My friend and I achieved similar results before when we are doing research on tapo C200, the low framerate issue is a real mystery :/
Here are what we did: https://drmnsamoliu.github.io/video.html
And just FYI there is an auth bypass vulnerability in their implementation of RTSP protocol, but we failed to receive anything useful too.

Any update on this? I also failed to fix the issue of low fps in my own research... ;/

i'm checking the ts files in order to see if we can fix this... it seems that there are two streams, video and i suposse an audio stream, but my ffmpeg doesn't recognice the second stream codec...
[mpegts @ 0x5623237e0600] probed stream 1 failed
[mpegts @ 0x5623237e0600] Could not find codec parameters for stream 1 (Unknown: none ([144][0][0][0] / 0x0090)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from '2021-12-10 13_21_05.ts':
Duration: 00:00:41.66, start: 1.934333, bitrate: 809 kb/s
Program 1
Stream #0:0[0x44]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuvj420p(pc, bt709, progressive), 1920x1080, 15 fps, 15 tbr, 90k tbn, 30 tbc
Stream #0:1[0x45]: Unknown: none ([144][0][0][0] / 0x0090)
i think its a subtitle stream... i tried removing it (just in case the subtitles stream is forcing some kind of weird fps), with no luck.
also, i've transformed each frame to jpg, and it seems that, even if the FPS are 15, the frames are repeated, so my guess is that the transceive method is making something wrong, or the json payload must have more params or some of the params must be changed in order to get the correct TS.

Has there been any more progress? I was wondering if someone has compared the original file on the SD-card and the one captured from the stream?

I have no idea if this could affect the dropping of frames, but when viewing in the app. It is possible to select different playback speeds. I guess having a faster playback speed means dropping frames. So maybe it is possible to experiment with different playback speeds to see if this affect the fps.

I have no idea if this could affect the dropping of frames, but when viewing in the app. It is possible to select different playback speeds. I guess having a faster playback speed means dropping frames. So maybe it is possible to experiment with different playback speeds to see if this affect the fps.

Good idea, unfortunately already tried it when originally implementing. It is passed as a parameter "scale": "1/1",.

i tried also modifying the scale param with every value I was capable of
thinking and the results remained the same: crappy video.
El mar, 25 ene 2022 a las 0:09, Juraj Nyíri @.***>)
escribió:

I have no idea if this could affect the dropping of frames, but when
viewing in the app. It is possible to select different playback speeds. I
guess having a faster playback speed means dropping frames. So maybe it is
possible to experiment with different playback speeds to see if this affect
the fps.
Good idea, unfortunately already tried it when originally implementing. It
is passed as a parameter "scale": "1/1",.
—
Reply to this email directly, view it on GitHub
https://github.com/JurajNyiri/pytapo/issues/4#issuecomment-1020641095,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYUS6IRHP7S43H7ILD43QLUXXLZVANCNFSM4TXB6JQA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

I have no idea if this could affect the dropping of frames, but when viewing in the app. It is possible to select different playback speeds. I guess having a faster playback speed means dropping frames. So maybe it is possible to experiment with different playback speeds to see if this affect the fps.

Good idea, unfortunately already tried it when originally implementing. It is passed as a parameter "scale": "1/1",.

Ahh to bad. :(
Whe Tapo care is activate, it says the videos is uploaded to the cloud, maybe by analyzing this traffic. Its possible to get more knowledge i to how videos are streamed/downloaded from the camera. Maybe this has already been drone. :/

A recent vulnerability was discovered where a reverse shell can be made by using post request (similar to what pytapo does). Maybe someone with version below 1.1.16 can utilize this to explore the inner workings of tapo C200. Unfortunately I have already updated my firmware to 1.1.18 before discovering this.
https://www.hacefresko.com/posts/tp-link-tapo-c200-unauthenticated-rce

One step forwards, one step back.
Discovered a way how to "turn off" encryption of stream (and get state of the encryption). As I later found out this simply changes authorization to stream from using cloud password to using a hardcoded super secret password.
After getting a recorded file with this way, I still get the same result as before after converting it with ffmpeg. I guess this at least confirms we are decoding the streamed data correctly.
The changes are in dev branch as always.

I analyzed chunks of data received when downloading the stream. There is always a bigger chunk, and 2 or 3 smaller chunks between the large ones. They always have length 376 decrypted, 384 encrypted but different content.
I tried ignoring these chunks when building the file, and interestingly, it resulted into exactly the same result as before, with smaller file size (4mb vs 4.4mb). So it looks like these chunks can be ignored and we achieve the same result. Maybe it is sound or I was also thinking of the "difference" between the big chunks (that seems to be missing in our resulting file) but there is not more of them on motion in file, always just 2-3.

Updated DEV branch with code changes adding ability to record live stream.
It uses the same port and communication as recordings, with following differences:

There is no need to send acknowledgments
Server does not return seq number

How to run:
from pytapo import Tapo
import json
import asyncio
import logging

logging.basicConfig(level=logging.DEBUG)
# change variables here
host = ""  # change to camera IP
user = ""  # your username
password = ""  # your password
password_cloud = ""  # set to your cloud password

tapo = Tapo(host, user, password, password_cloud)

devID = tapo.getBasicInfo()["device_info"]["basic_info"]["dev_id"]

async def download_async():
    print("Starting...")
    mediaSession = tapo.getMediaSession()
    async with mediaSession:
        payload2 = {
            "params": {
                "preview": {
                    "audio": ["default"],
                    "channels": [0],
                    "deviceId": devID,
                    "resolutions": ["HD"],
                },
                "method": "get",
            },
            "type": "request",
        }

        payload = json.dumps(payload2)
        output = b""
        dataChunks = 0
        async for resp in mediaSession.transceive(payload):
            if resp.mimetype == "video/mp2t":
                # if len(resp.plaintext) != 376:
                output += resp.plaintext
                print(len(resp.plaintext))
                dataChunks += 1
            if dataChunks > 2000:
                break

        fileName = "./output/stream.ts"
        print("Saving to " + fileName + "...")
        file = open(fileName, "wb")
        file.write(output)
        file.close()

loop = asyncio.get_event_loop()
loop.run_until_complete(download_async())

The resulting file is broken the same way as recordings are.
Thanks to this we know:

its not an issue with seq or acknowledgements.
its not an issue with parameters sent

With that, we can deduce that it can be an issue with:

Incorrect codec conversion via ffmpeg of resulting ts file
Incorrect data gathering in script
Something else?

I'm not getting anything. zero size all the time.
python 3.10.8
tapo C200 firmware 1.1.18
(venv) [jepes@ARCHJEPES pytapodev]$ rm output/stream.ts;  python vidtest.py && ls -lha output/stream.ts
rm: cannot remove 'output/stream.ts': No such file or directory
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST / HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=2b9780464aa92c3541d432cec0fa4dc1/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=2b9780464aa92c3541d432cec0fa4dc1/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=2b9780464aa92c3541d432cec0fa4dc1/ds HTTP/1.1" 200 None
/home/jepes/Scripts/pytapodev/vidtest.py:54: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
DEBUG:asyncio:Using selector: EpollSelector
Starting...
INFO:pytapo.media_stream.session:Connected to the media streaming server
DEBUG:pytapo.media_stream.session:Authentication data retrieved
AES key: e38abba4bdc01ea2d70432e4c06ab103, iv: 97f17abbc57362b5a3b13f668a548bf4
DEBUG:pytapo.media_stream.crypto:AES cipher set up correctly
DEBUG:pytapo.media_stream.session:AES key exchange performed
DEBUG:pytapo.media_stream.session:Plaintext request of type application/json sent (sequence 23309, session None), expecting 51 responses from queue<PHONE_NUMBER>95200
DEBUG:pytapo.media_stream.session:Response handler is running
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Plaintext response of type application/json processed (sequence 23309, session 4), dispatching to queue<PHONE_NUMBER>95200
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Got one response from queue<PHONE_NUMBER>95200
DEBUG:pytapo.media_stream.session:Server did not send a new chunk in 1.0 sec (sequence 23309, session 4), assuming the stream is over
Saving to ./output/stream.ts...
-rw-r--r-- 1 jepes jepes 0 Nov 25 16:34 output/stream.ts
(venv) [jepes@ARCHJEPES pytapodev]$ rm output/stream.ts;  python vidtest.py && ls -lha output/stream.ts
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST / HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=ecb7644838c447a2306b38017a63c01b/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=ecb7644838c447a2306b38017a63c01b/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=ecb7644838c447a2306b38017a63c01b/ds HTTP/1.1" 200 None
/home/jepes/Scripts/pytapodev/vidtest.py:54: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
DEBUG:asyncio:Using selector: EpollSelector
Starting...
INFO:pytapo.media_stream.session:Connected to the media streaming server
DEBUG:pytapo.media_stream.session:Authentication data retrieved
AES key: 0cd3d919492e7887a2811119630e63bb, iv: 5a7a33a3a5884d1f9978b7b12fcf2b3a
DEBUG:pytapo.media_stream.crypto:AES cipher set up correctly
DEBUG:pytapo.media_stream.session:AES key exchange performed
DEBUG:pytapo.media_stream.session:Plaintext request of type application/json sent (sequence 7707, session None), expecting 51 responses from queue<PHONE_NUMBER>60976
DEBUG:pytapo.media_stream.session:Response handler is running
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Plaintext response of type application/json processed (sequence 7707, session 5), dispatching to queue<PHONE_NUMBER>60976
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Got one response from queue<PHONE_NUMBER>60976
DEBUG:pytapo.media_stream.session:Server did not send a new chunk in 1.0 sec (sequence 7707, session 5), assuming the stream is over
Saving to ./output/stream.ts...
-rw-r--r-- 1 jepes jepes 0 Nov 25 16:34 output/stream.ts
(venv) [jepes@ARCHJEPES pytapodev]$
(venv) [jepes@ARCHJEPES pytapodev]$ rm output/stream.ts;  python vidtest.py && ls -lha output/stream.ts
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST / HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=5fea787fb38b42d5a1cf01e62cc1c0ba/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=5fea787fb38b42d5a1cf01e62cc1c0ba/ds HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): <IP_ADDRESS>:443
DEBUG:urllib3.connectionpool:https://<IP_ADDRESS>:443 "POST /stok=5fea787fb38b42d5a1cf01e62cc1c0ba/ds HTTP/1.1" 200 None
/home/jepes/Scripts/pytapodev/vidtest.py:54: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
DEBUG:asyncio:Using selector: EpollSelector
Starting...
INFO:pytapo.media_stream.session:Connected to the media streaming server
DEBUG:pytapo.media_stream.session:Authentication data retrieved
AES key: d73cc07cfdf4c9bf3256a036ee192241, iv: 792a20ca67719a583ab44c84bfb3772b
DEBUG:pytapo.media_stream.crypto:AES cipher set up correctly
DEBUG:pytapo.media_stream.session:AES key exchange performed
DEBUG:pytapo.media_stream.session:Plaintext request of type application/json sent (sequence 18349, session None), expecting 51 responses from queue<PHONE_NUMBER>50720
DEBUG:pytapo.media_stream.session:Response handler is running
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Plaintext response of type application/json processed (sequence 18349, session 6), dispatching to queue<PHONE_NUMBER>50720
DEBUG:pytapo.media_stream.session:Handling new server response
DEBUG:pytapo.media_stream.session:Got one response from queue<PHONE_NUMBER>50720
DEBUG:pytapo.media_stream.session:Server did not send a new chunk in 1.0 sec (sequence 18349, session 6), assuming the stream is over
Saving to ./output/stream.ts...
-rw-r--r-- 1 jepes jepes 0 Nov 25 16:34 output/stream.ts
(venv) [jepes@ARCHJEPES pytapodev]$

@jepes1981 please come to discord to debug the issue with running this script.

There has been progress on this made on go2rtc repository.
I have spent a few hours on this comparing the code and have found that we need to be decoding the received payload.
So here:
async for resp in mediaSession.transceive(payload):
            if resp.mimetype == "video/mp2t":
                # if len(resp.plaintext) != 376:
                output += resp.plaintext
                print(len(resp.plaintext))
                dataChunks += 1
            if dataChunks > 2000:
                break

We need to be decoding resp.plaintext as a TS packet, similarly to how it is done here.
I have verified that:

pytapo & go2rtc is receiving the same communication
pytapo & go2rtc is is receiving exactly the same sizes of payloads before decoding
pytapo & go2rtc is is receiving exactly the same sizes of payloads after decoding

The difference between is that pytapo is not decoding the received packets, where as go2rtc is doing that and sending the decoded payload, which is significantly smaller. There is also something happening with tracks there, I am not sure what yet.
I have tried following libraries in python to achieve the same with no success:

threefive
RTP
pyrtp
digestpacket function

Discovery
Modified go2rtc with following logs:

This gives me following output:

Notice the 0 0 1, this is important part of the data and needs to be in every chunk received.
Implementation
Next, I have rewritten all the methods from go2rtc into pytapo (ouch) as described in above comment. File attached. I think they should be working as expected but have not tested them fully yet.
tmp6.py.zip
Result
I have found that when using the functions that I wrote (if uncommented in code above/screenshot below), I land in a code branch where go2rtc never lands.

The reason for that is that the chunks that I am receiving in pytapo do not have 0 0 1 like the chunks in go2rtc.

Simplifying debugging for above

Download:
tmp6.py.zip
Just fill in ENV variables at the start of file or hardcode them, you can use the latest github main branch version.
Go2RTC output

PyTapo output

Next action item
We now have an easily debuggable scenario with comparison code and we need to make sure that the output of pytapo is the same as Go2RTC.

SOLVED AFTER MORE THAN 2 YEARS
All we needed to do was refresh IV in crypto after every message.

Also, all that code I rewrote from go to python for stream RTP was not needed in the end.

Coming soon.

Script to download all recordings have been added here.
You can launch it like so:
output="/Users/jurajnyiri/tapo/recordings/LivingRoom" date=20230218 host="<IP_ADDRESS>" user="myUser" password="s3cr3t" password_cloud="v3rys3cr3t" python3 DownloadRecordings.py

All needed functionality has been added. Recordings now also include audio converted into AAC.
Instructions are in readme.


when I login，show sth wrong

info:
/Users/yubingjie/.oh-my-zsh/tools/check_for_upgrade.sh:3: failed to load module zsh/datetime': dlopen(/usr/lib/zsh/5.0.8/zsh/datetime.so, 9): image not found _current_epoch:1: bad math expression: operand expected at / 60 / 60 ...'
Here is some commands I tried
which -a zsh
/usr/local/bin/zsh
/bin/zsh
echo $path
/usr/local/bin /usr/bin /bin /usr/sbin /sbin
Also I have tried uninstall oh-my-zsh and re-installed,but the problem still occcured

Duplicate of #5438, please subscribe to that issue and follow the solutions suggested there.


Allow newlines and other shell metacharacters in deploy:script: and o…

…ther DPL arguments
https://travis-ci.community/t/deploy-scripts-not-supporting-multiline-strings/8297
Considered alternatives:

eval the values encoded with %p in DPL -- unsafe
use JSON encoding -- unnecessary and still need to escape shell metacharacters in the result

Thank you!

Thank you!

Looks like this is causing https://travis-ci.community/t/s3-aws-deploy-access-key-id/10967.
I'll revert it.

Looks like this is causing https://travis-ci.community/t/s3-aws-deploy-access-key-id/10967.
I'll revert it.


js alerts on webview are silently ignored

This breaks pages/sites that use alerts, such as confirmations before a delete and similar.
According to this answer the webview should use a delegate to make it work. I am not versed in iOS, so I am not sure if that is the proper solution.

Please could you kindly provide some sample json in order to test this behaviour.
thanks 👍


[core] Avoid tag createTime not being set when timeRetained is null

Purpose

Linked issue: close #3556

Tests

API and Format
No
Documentation
No

See https://github.com/apache/paimon/issues/3556


Dynamically manipulate ClayItem at run-time

I think Clay is missing the option to dynamically manipulate a ClayItem at run-time.
For example:
The user needs to login to a server, then the settings page fetches some data from the server.
According to the fetched data I need to manipulate the options property of a select ClayItem.
I would wish, that there is some kind of rebuild function available for ClayItems
For example
var roomSelect = clayConfig.getItemByAppKey("room");
var options = roomSelect.config.options;
var rooms = functionToGetRooms();
rooms.forEach(function(room, idx, arr) {
         options.push({
                label: room.name,
                value: room.uuid
        })
});
roomSelect.config.options = options;
roomSelect.rebuild(); //-> Rebuilds the ClayItem and also updates the DOM

As a current workaround I am manually appending the options to the select:
var roomSelect = clayConfig.getItemByAppKey("room");
var rooms = MSComp.getRooms(struct);
rooms.forEach(function(room, idx, arr) {
        var option = HTML('<option value="' + room.value + '" class="item-select-option">' + room.name + '</option>');
        roomSelect.$manipulatorTarget.add(option);
});

The tricky part is making sure events, etc keep firing as one would expect. in the mean time Clay now supports the entire page being rebuilt with clayConfig.build()

Something like this would work as an interim solution.

function() {
  var clayConfig = this;

  function onBtnClick() {
    clayConfig.config[1].items[1].options = [
      { "label": "Dynamic 1", "value": "1" },
      { "label": "Dynamic 2", "value": "2" },
      { "label": "Dynamic 3", "value": "3" }
    ]

    clayConfig.build();
  }

  clayConfig.on(clayConfig.EVENTS.AFTER_BUILD, function() {
    clayConfig.getItemById('triggerBtn').on('click', onBtnClick);

    clayConfig.getItemById('selectionGroup').on('change', function() {
      clayConfig.getItemById('textDisplay').set(this.get());
    });
  });
}


Improve design of overlay frames

The design should be improved, e.g. transparent background and smaller icons, maybe no borders of entries etc.

Use rgb(90, 90, 90) as background color.


Add support for star-rating

It would be nice to support non-jquery star rating:
https://pryley.github.io/star-rating.js/
I'm open to make a PR for this, my goal is to get rid of the jQuery dependency ;-)

Hello! Yes, feel free to create a pull request.

Closing this since it exists in core Rating question


Change coverage (#146)

Codecov Report

Merging #162 (0861d57) into main (748614b) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #162   +/-   ##
=======================================
  Coverage   84.68%   84.69%
=======================================
  Files          80       80
  Lines        2397     2398    +1
=======================================
+ Hits         2030     2031    +1
  Misses        367      367

Impacted Files
Coverage Δ

lib/src/code/code.dart
93.58% <100.00%> (+0.04%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Possible message rendering bug

[Not sure if this has been filed!]
Use the demo project with extra messages. When pushing via storyboard or programmatically, there seems to be some weird animation of cells moving from left to the right. (In fact, I believe it's just a scroll to bottom issue). This effect is more noticeable when the last few messages in the view are sent by the application user.
In my application, the effect is much more visible as compared to the demo project. I have attached two videos for you to see.
Currently, I have enabled animation in viewWillAppear: at line 232:
[self scrollToBottomAnimated:YES];

See the video attached.
(Some things you can't unsee bro!)
Apparently, I can't upload the videos here. Please refer to the dropbox link: https://www.dropbox.com/sh/nrjo1h9fo762d16/AACISmQwR13Ktzlx0yCuubo4a?dl=0.
Thanks very much.

Hello everyone!
I'm sorry to inform the community that I'm officially deprecating this project. 😢 Please read my blog post for details:
http://www.jessesquires.com/blog/officially-deprecating-jsqmessagesviewcontroller/
Thus, I'm closing all issues and pull requests and making the necessary updates to formally deprecate the library. I'm sorry if this is unexpected or disappointing. Please know that this was an extremely difficult decision to make. I'd like to thank everyone here for contributing and making this project so great. It was a fun 4 years. 😊
Thanks for understanding,
— jsq


Add basic documentation

Objective
Add the necessary documentation related to the project

Description
The mentioned documentations should cover

Issues/stories tags
Project description adding column states descriptions
Add documentation about selected git branching strategy

Steps

[x] Create issue tags related to possible scenarios (like documentation, research, feature, bug, enhancement)
[x] Create an issue template for component creation
[x] Learn about best practices for ticket workflow
[x] Add description related to column status and roadmap main points
[x] Learn about possible git branching strategies and document the selected choice

Ticket relationships

Relationship
Involved tickets

Is child from
none

In parent of
none

Is related to
#1 - #3

Is blocked by
none

It blocks
none

Technical notes
none

To understand how to integrate my gitflow with semantic-release in a smooth way, I need to get into library documentation before close the ticket

The uploaded image is the associated documentation of the first gitflow draft. This poposition will be tested in issue named Create an test npm package in order to get the best possible branching strategy which will help an automated way of publish the package in Github and npm with same version number.


Fix repo

Which issue(s) this PR fixes:

Issue #
What this PR does / why we need it:
The current repo is buggy. Running make will error out.
Special notes for your reviewer:
Additional documentation or context

Renaming repo to cli should update the code. This PR just updates the necessary parts. For the rest, I am not sure if we can retain them. I mean, continue using name longhorn-preflight


Disable default values?

I'm not sure if this is the right place to ask, or is it even possible, but:
Would there be a way NOT to take the default values provided by the Terraform provider?
We are using this Pulumi provider with Go, and redefine some of the defaults to fit our environment, but not all, an those interfere with our code eventually.
For example:
Lets have a VM template we want to clone. The template already has some disks. If I do not define the disks, the default size would apply (8G) which is lower then what's already in the template, thus Proxmox fails that it cannot shrink a disk.
This can be done with the GUI: one can just clone a template VM with only providing a name for the new VM.

thanks for your input. unfortunately, this is how Pulumi (bridge) works - it takes the defaults from the underlying Terraform provider.
i see this as a more generic input for the Pulumi team. the right place will be either Pulumi directly, or the bridge used for Terraform providers.


Modules/kwin: add window titlebar button config

Draft since since you can add any string you want even if it would generate a config that isn't valid

@magnouvean should I leave it as is?
I'm thinking of either having something closer to what kwinrc actually has, but verifying that the options add buttons that exist ( so how it is rn but with checking):
programs.plasma.kwin = {
  titlebarButtons.left = [ "S" "F" ];
  titlebarButtons.right = [ "H" "I" "A" "X" ];
};

or something that's longer to type, but that uses the same names as the system settings does:
programs.plasma.kwin = {
  titlebarButtons.left = [ "on-all-desktops" "keep-above-windows" ];
  titlebarButtons.right = [ "help" "minimize" "maximize" "close" ];
};

I'd vote for longer names, so the config is self-documenting (no need for an additional doc to refer to)

I do agree with @bew as this would be a higher-level so we may as well add an interface that makes more sense, as the default plasma config-file options aren't that helpful. Our aim for the higher-level interfaces should be closer to system-settings than config-files imo.

Got it, long names it is!

Spent quite a while working on it, but I can't quite get it to work ( it doesn't respect the order you specify the buttons to be in)
I'll give it another look tomorrow
I didn't want to push know-bad code here but I do need help for this so I did a patch with my changes
wip.zip

Right, I'll try to take a look it later today if you haven't figured it out by then :)

Figured it out, I don't think it's the cleanest way of doing it but it works


Add Pullover 0.3.2

Add new cask definition for Pullover based on contributing docs.

Updated.

Thank you for the contribution. It was merged directly as 6cc48598e72655048664c5b609c348b6035528e0 to keep commit history cleaner. Your contribution is still credited to you.