Get channel name when RTM_EVENTS.MESSAGE

Hey team!
I'm migrating from 1.4.1 to 2.3.0.
I'm using something like this:
slack.on('message', function(message) {
    if(this.getChannelGroupOrDMByID(message.channel).name == 'a_random_channel_name') {
        // Do something
    }
});

Is there a new way to get a channel name (like getChannelGroupOrDMByID('XXXXXX').name) from a message? I don't find it in documentation.
Thank you for your help,
Rémi.

Yup, you'll need to use a dataStore to get access to those helper functions now:
const RtmClient = require('@slack/client').RtmClient;
const CLIENT_EVENTS = require('@slack/client').CLIENT_EVENTS;
const MemoryDataStore = require('@slack/client').MemoryDataStore;

let slack = new RtmClient(YOUR_TOKEN_HERE, {
  logLevel: 'error',
  dataStore: new MemoryDataStore(),
  autoReconnect: true,
  autoMark: true
});

slack.on(RTM_EVENTS.MESSAGE, (message) => {
  let channel = slack.dataStore.getChannelGroupOrDMById(message.channel);

  console.log(channel);

  slack.sendMessage(`Hello ${channel.name}!`, channel.id);
});

It works!
Really thank you for your fast answer :heart:


[WIP] allow dynamic creation & configuration of stylis per stylesheet

Closes #719 #844
If this works the way I think it will, the stylis stringifier is now dynamically created with the new stylesheet and can be replaced at runtime via use of StylesheetManager as a React context provider.

Can we add a test to make sure RTL support is possible with this patch?

Definitely. I want to get all the canary tests passing first before merging this so I have some assurance I'm not accidentally breaking anything.

I want to get all the canary tests passing first before merging this so I have some assurance I'm not accidentally breaking anything.

Good call.

@kitten @mxstbr ok this should be ready to go. added tests for stylis options and plugins

Also ran benchmarks and there's no change 👍


How to get skipoffset in Linear (vast).

Help me. Thanks!!!
How to get skipoffset in Linear.

Hi!
I have same problem.
Here is XML I'm trying to load:
https://drive.google.com/open?id=14-tz3VkpO7VQeG0YGma2fUlTpKE38hS3
How can i get skipoffset attribute value ?


I faced 404 That model does not exist

Hello there, I faced the error message, any clue thanks
https://api.openai.com/v1/completions
{
  model: 'text-chat-davinci-002-sh-alpha-aoruigiofdj83',
  temperature: 0.8,
  top_p: 1,
  presence_penalty: 1,
  stop: [ '<|im_end|>', '<|im_sep|>', '\nUser:' ],
  max_tokens: 1024,
  prompt: '\n' +
    '<|im_sep|>Instructions:\n' +
    'You are ChatGPT, a large language model trained by OpenAI.\n' +
    'Current date: February 28, 2023<|im_sep|>\n' +
    '\n' +
    'User:\n' +
    'Hello, how are you today?<|im_end|>\n' +
    'ChatGPT:\n'
}

Error: Failed to send message. HTTP 404 - {
    "error": {
        "message": "That model does not exist",
        "type": "invalid_request_error",
        "param": null,
        "code": null
    }
}

    at ChatGPTClient.getCompletion (file:///Users/safasdf/workspace/node-chatgpt-api/src/ChatGPTClient.js:172:27)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async ChatGPTClient.sendMessage (file:///Users/safasdf/workspace/node-chatgpt-api/src/ChatGPTClient.js:232:22)
    at async Object.<anonymous> (file:///Users/safasdf/workspace/node-chatgpt-api/bin/server.js:113:18) {
  status: 404,
  json: {
    error: {
      message: 'That model does not exist',
      type: 'invalid_request_error',
      param: null,
      code: null
    }
  }
}

This model has been patched/deleted, so you'll need to use text-davinci-003 (which it's paid) or use  ChatGPTBrowserClient instead. (you have to provide accessToken else it won't work)

Duplicate of #115


leveldown-hyper should be a fork of node-leveldown

It's a bit messy right now since the leveldown-hyper module on npm corresponds to the hyper-leveldb branch in this repository. Also, we are using a git submodule for the HyperLevelDB dependency.

:+1:

+1

+1
Il giorno sab 28 mar 2015 alle 03:34 Erik Kristensen <
<EMAIL_ADDRESS>ha scritto:

+1
—
Reply to this email directly or view it on GitHub
https://github.com/rvagg/node-leveldown/issues/159#issuecomment-87146874
.

:+1:

https://github.com/Level/leveldown-hyper


UI loaded, but no data

Hello and Guten Tag,
after I bought a second hand ID.3 1st and the "weconnect" packages from VW are activated, I wanted to show data of the car on the MagicMirror, because not every one likes to check the car via the smartphone app. I think that everything went fine, but there is no data shown on the UI.
Data connection via the VW app is working.
I use MagicMirror 2.26.0 on a Raspi 3+. OS is Raspberry Pi OS 32-Bit Legacy from 2023. Python was installed 3.9.2 and I installed like in your doc mentioned 3.9.15.
Please help @NikolasRupp where can I check, whats wrong?
Thanks and regards!

So is there nothing to see on the Screen or how does it Look Like when it's running? Could you maybe send me a picture of what it looks Like, if there is anything?

Screenshot: Bottom/Left the weconnect part

So the Problem seem to be that you run it on  32-Bit.
I made a small change. Could you please pull it again and then retry it.
You won't be able to see the timestamp from the last update of the car. I have to find a solution for that. But i can't really test it at the moment because i don't have a rpi and can only run it in an windows envirenment which has 64-Bit. My rpi will arrive later this week. Maybe I will then have time to test it

Okay this seems strange. I never came across this. I think you live in Austria right? Unfortunately I can only test this for germany.
If youl ive in Autria do you login in web via
https://www.myvolkswagen.net/at/de/myvolkswagen.html
or
https://www.volkswagen.de/de/besitzer-und-nutzer/myvolkswagen.html?
Maybe this is the Problem. But I will have a deeper look into this once my rpi is here

Okay sorry it seems this does not come from the vw api but from the google one. Do you use the "googleAPI" parameter. If so please try it without it

Yes, Google API caused the problem. Deleted the API String and everything okay! Nice! Really don't need the exact position of the car. Thanks for the help!
Grüße aus Niederösterreich!

Okay perfect. Happy to help.


app hosted using manual deploy is working fine. But another page in the app is throwing 403

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
d3l40h67b0qp6q
Region
us-east-1
Amplify Console feature
Manual deploys
Describe the bug
I have hosted my app using amplify by uploading the app's build folder (contents only not the folder) zip file.
I'm able to access my app's home page and other functionalities are working fine. But when I tried to navigate to other page in my app domain, it's throwing 403
working fine - https://prod.d3l40h67b0qp6q.amplifyapp.com/
Family advisory panel - https://prod.d3l40h67b0qp6q.amplifyapp.com/advisory/
(The above link is working absolutely fine when deployed in a EC2 instance and started the server using npm start)
Failed request :
curl 'https://prod.d3l40h67b0qp6q.amplifyapp.com/advisory/'
-H 'authority: prod.d3l40h67b0qp6q.amplifyapp.com'
-H 'upgrade-insecure-requests: 1'
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9'
-H 'sec-fetch-site: same-origin'
-H 'sec-fetch-mode: navigate'
-H 'sec-fetch-user: ?1'
-H 'sec-fetch-dest: document'
-H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"'
-H 'sec-ch-ua-mobile: ?0'
-H 'referer: https://dev.safem8.app/'
-H 'accept-language: en-IN,en;q=0.9,en-GB;q=0.8,en-US;q=0.7,te;q=0.6'
-H 'cookie: _ga=GA1.2.1360142473.1627388077; _gid=GA1.2.1157002783.1629373321; crisp-client%2Fsession%2F27bc4e54-cc62-4088-80dc-2225afdf04df=session_715b2b83-580b-4ecc-8356-546616dc1cf6'
--compressed
Expected behavior
Expecting proper 200 response for the https://prod.d3l40h67b0qp6q.amplifyapp.com/advisory/ url
Reproduction steps
Go to https://prod.d3l40h67b0qp6q.amplifyapp.com and click on family advisory panel link
Build Settings
No response
Additional information
No response

The fix suggested in this link worked for me. But didn't understand the root cause of the issue.
https://victorleungtw.medium.com/fix-aws-amplify-angular-app-error-on-access-denied-error-73c9476f9552


Bad Request when expanding on thumbnails for driveItems delta on AAD.

Category

[ ] Question
[ ] Documentation issue
[X] Bug

Expected or Desired Behavior
According to https://docs.microsoft.com/en-us/graph/api/driveitem-delta?view=graph-rest-1.0&tabs=http#optional-query-parameters drive item delta queries support $expand:

Optional query parameters
This method supports the $select, $expand, and $top OData query parameters to customize the response.

I'm expecting this request to succeed (regardless of whether I login with my AAD or MSA account):
https://graph.microsoft.com/v1.0/me/drive/root/delta?$expand=thumbnails
Observed Behavior
200 for MSA and successfully returns thumbnails, 400 for AAD.
{
    "error": {
        "code": "badArgument",
        "message": "One of the provided arguments is not acceptable.",
        "innerError": {
            "request-id": "2a0afa55-01c4-4dd0-9c60-19a04b7ceece",
            "date": "2020-05-20T01:18:34"
        }
    }
}

Steps to Reproduce
Login with a work/school account and GET https://graph.microsoft.com/v1.0/me/drive/root/delta?$expand=thumbnails
Repro on both v1.0 and beta endpoints of Graph.
Thank you.

Could you please clarify whether this is happening for an MSA account through Graph, or if this is a issue with ODB/work/school through Graph?  Thumbnails through delta are not supported in ODB.

@chackman I'd already specified both in the subject & descriptin that this is happening on AAD account (work/school) and works fine on MSA. What's ODB ? Could you update documentation because clearly this information is no where to be found.

"ODB" stands for OneDrive for Business. This is on our documentation backlog.

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


Update osgi compendium and osgi core from 5.0.0 to 6.0.0

OSGI 6 and DS 1.3 have a few new features that improve code quality and readability:

Reference Annotation supports field as target (https://docs.osgi.org/javadoc/r6/cmpn/index.html?org/osgi/service/component/annotations/Reference.html)
Configuration Property Types (http://blog.vogella.com/2016/09/26/configuring-osgi-declarative-services/ & http://njbartlett.github.io/2015/08/17/osgir6-declarative-services.html)

Reference Annotation Example
public class ToggleMinimumCardinalityCommand {

    @Reference
    ConfigurationAdmin admin;
 }

Configuration Property Types
public @interface MessageConfig {
    String message() default "";
    int iteration() default 0;
}

public class StaticConfiguredComponent {

    @Activate
    void activate(MessageConfig config) {
        String msg = config.message();
        int iter = config.iteration();

        for (int i = 1; i <= iter; i++) {
            System.out.println(i + ": " + msg);
        }
    }
}

[x] have a concise title
[x] close an accompanying issue if one exists
[x] be against the correct branch
[x] include migration scripts and documentation, if appropriate
[x] pass automated tests
[x] have a clean commit history
[x] have proper commit messages (title and body) for all commits

Requesting Lars takes a look at this since he's working on (afaik) the Karaf update too.


Alias paths aren't being resolved by language server

Describe the bug
Not being able to go to an implementation of a function when using an alias like default $lib or a custom one;
Suggestions for typing imports don't work.

Reproduction
Repo with reproduction - https://github.com/Joonel/alias-bug
Create a new empty project, no TS, no JSDoc.
Add any @custom-alias for a folder in svelte.config.js, import a function on +page.svelte from the index.js in that folder, then try to go to the implementation of that function.
svelte.config.js:
  kit: {
    adapter: adapter(),
    alias: {
      "@custom-alias": "./src/custom-alias",
    },
  },

paths in ./svelte-kit/tsconfig.json are as expected:
"compilerOptions": {
"paths": {
	"@custom-alias": [
		"../src/custom-alias"
	],
	"@custom-alias/*": [
		"../src/custom-alias/*"
	],
	"$lib": [
		"../src/lib"
	],
	"$lib/*": [
		"../src/lib/*"
	]
},

Logs
No response
System Info
System:
    OS: Windows 11 10.0.22631
  Binaries:
    Node: 20.10.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.4.0 - C:\Program Files\nodejs\npm.CMD
    pnpm: 9.0.6 - ~\AppData\Roaming\npm\pnpm.CMD
  Browsers:
    Edge: Chromium (123.0.2420.97)
    Internet Explorer: 11.0.22621.3527
  npmPackages:
    @sveltejs/adapter-auto: ^3.2.1 => 3.2.1
    @sveltejs/kit: ^2.5.10 => 2.5.10
    @sveltejs/vite-plugin-svelte: ^3.1.0 => 3.1.0
    svelte: ^4.2.17 => 4.2.17
    vite: ^5.2.11 => 5.2.11

Severity
serious, but I can work around it
Additional Information
No response

You need to add a jsconfig.json referencing that generated tsconfig, else intellisense doesn't know what these references are. We should adjust the npm create svelte CLI to add a minimum jsconfig.json if you chose no for both JsDoc and TS.


Another wat to match tags for CDATA

I suspect that we only want to match exact field names like "ldap_authcn" and so on anyway, so this code could remove the substr() stuff.
I left the "descr" and "detail" clauses, because maybe it is intended to match "descr1" "detail01" and stuff like that also.

See #3708 for minimal fix to substr() counts.
This is a possible alternate way to go to refactor and fix.

I believe the original idea was to protect any strings starting with these names, rather than the names exactly. To reduce the length of the list overall. Though I'm not sure how much we leverage that in practice, I'd rather not risk breaking something that has been working. Fixing the individual tests seems like the better fix to me.

Yes, I agree. Making a minimal change here for now is the safest thing to do. Then refactor/improve later.


Possible memory leak adding many background download requests in a loop

Excuse the last post - accidentally hit something...
I have a strange issue with Instruments leaks. I'm running a loop to add many download requests to a background session. The SessionManager is contained in a class singleton pretty much as follows:
class BackgroundTransferCoordinator {
static let sharedInstance = BackgroundTransferCoordinator()
lazy var session: SessionManager = {
// session with background configuration
}()
// .. too much to show
}
When building background requests in a loop, the following code results in a stunning number of leaks after adding just four requests:
for _ in 0..<4 {
let request = self.session.download(url)
request.resume()
}
To confuse matters, adding less than 4 items is more than often OK. Replace the for loop content with this:
let request = self.session.session.downloadTask(with: url)
request.resume()
...and you can add as any requests as you like, with no leaks showing up. I'm typically adding 50 or more requests on a first run and then adding a couple per day thereafter - documents.
The BackgroundTransferCoordinator class sets handlers for:
session.delegate.taskDidComplete
session.delegate.downloadDidFinishDownloadingToURL
I actually run the code inside the getAllTasks completion handler to query existing background tasks and then add missing tasks. However, other code builds requests on the main queue and that also fails after adding more than three in a loop. There is no crash occurring, and all requests complete properly - got the files to prove it.
As far as I can see, the difference appears to be the second example, by using the URLSession directly, bypasses this:
private func download( _ downloadable: DownloadRequest.Downloadable,
to destination: DownloadRequest.DownloadFileDestination?)
-> DownloadRequest
... and requests do not end up being added to SessionDelegate.requests
I'm not convinced this isn't an issue with Instruments or the way I'm using it. I will try to make a test project in the next couple of days if possible.

Can you post the leaks output or the lines it points to as responsible? While a demo project would be appreciated, being able to examine the leaks you've seen would be helpful in the meantime.

I have attached a couple of zipped CSV files. I have also confirmed the issue exists adding requests in a loop though I have no clear idea yet why. The following code block shows the problem code. It is running inside the getAllTasks completionHandler but has exactly the same issue running on the main queue.
IQSMS-Reporting-leaksWithLoopIssue.csv.zip
IQSMS-Reporting-leaksNoLoopIssue.csv.zip

OK, safari is not playing ball with my code when using // comments. The following code picks up unreadable folder files and starts a background download. The TransferID().string is a short encoded description string used to identify the task later. I'm not keeping any references to download tasks. Session.startsRequestsImmediately  is false.
        if completed < total {
            let unreachable = folder.manifest.unreachable
            for rec in unreachable {
                let taskDescription = TransferID(type: .update, stagingId: stagingId, name: rec.filename, md5sum: rec.md5sum).string
                if taskDescriptions.contains(taskDescription) == false {

                    /*
                    let req = self.session.download(rec.url)
                    req.task?.taskDescription = taskDescription
                    req.resume()
                     */
                    let req = self.session.session.downloadTask(with: rec.url)
                    req.taskDescription = taskDescription
                    req.resume()

                }

            }
        } else if total > 0 {
            self.stagingID = 0
            _ = folder.convertToReady()
        }

There are 20 files in the code loop shown previously. I only have to loop for 4 to get the leaks trace. Each time the code run is repeated (deleting the folder of files), total memory used grows, so I'm beginning to  think block is being retained somewhere.

The loop code without the issue produces this image. I clicked refresh 10 times in the trace period. The error message on the App is because I faked an md5 checksum error on the last file in order to loop the test.

Also, I forgot to mention, there are 2 x SessionManagers running. The default is doing the foreground fetches very nicely thank you :-). That should explain why there are 2 x NSLock objects shown.

The lock leaks are a false positive. As for the background leaks, I'm not sure what's happening there, but it looks like you're leaking the world almost. If you can get a test project to us that would greatly help with our investigation.

I'll post a test later today.

AlamofireBackgroundSessionTest.zip
One test project!

Do try setting the loop to 0..<4

Ok guys, bit of code for you to look at. The mod is not in the best place - should be in the if taskDidComplete block but I think you will get the idea as to what is going wrong if session.taskDidComplete is wired up:
open func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {
    /// Executed after it is determined that the request is not going to be retried
    let completeTask: (URLSession, URLSessionTask, Error?) -> Void = { [weak self] session, task, error in
        guard let strongSelf = self else { return }

        if let taskDidComplete = strongSelf.taskDidComplete {
            taskDidComplete(session, task, error)
        } else if let delegate = strongSelf[task]?.delegate {
            delegate.urlSession(session, task: task, didCompleteWithError: error)
        }

        NotificationCenter.default.post(
            name: Notification.Name.Task.DidComplete,
            object: strongSelf,
            userInfo: [Notification.Key.Task: task]
        )

        // ADDED. delegate queue is not resumed if only Session.taskDidComplete is wired!
        if let delegate = strongSelf[task]?.delegate {
            delegate.queue.isSuspended = false
        }

        strongSelf[task] = nil
    }

    guard let request = self[task], let sessionManager = sessionManager else {
        completeTask(session, task, error)
        return
    }

    // Run all validations on the request before checking if an error occurred
    request.validations.forEach { $0() }

    // Determine whether an error has occurred
    var error: Error? = error

    if let taskDelegate = self[task]?.delegate, taskDelegate.error != nil {
        error = taskDelegate.error
    }

    /// If an error occurred and the retrier is set, asynchronously ask the retrier if the request
    /// should be retried. Otherwise, complete the task by notifying the task delegate.
    if let retrier = retrier, let error = error {
        retrier.should(sessionManager, retry: request, with: error) { [weak self] shouldRetry, timeDelay in
            guard shouldRetry else { completeTask(session, task, error) ; return }

            DispatchQueue.utility.after(timeDelay) { [weak self] in
                guard let strongSelf = self else { return }

                let retrySucceeded = strongSelf.sessionManager?.retry(request) ?? false

                if retrySucceeded, let task = request.task {
                    strongSelf[task] = request
                    return
                } else {
                    completeTask(session, task, error)
                }
            }
        }
    } else {
        completeTask(session, task, error)
    }
}

Oh, I forgot to mention, if you just move the 'private let lock = NSLock()' right to the top of the SessionDelegate class, you want see this anymore in Leaks.

In my case(Swift 3, iOS 10.2, Alamofire 4.0.1) I had to set the configuration like this to avoid memory leaks from CFNetwork library:
configuration.urlCache = nil
configuration.urlCredentialStorage = nil
configuration.requestCachePolicy = .reloadIgnoringLocalCacheData

The issue looks different from simulator to device. I ran the same code both on device and the simulator they look different. (Leak object) But when I look at the stack trace on both SessionDelegate.init() can be seen.

@unusuallyreticent
Oh, I forgot to mention, if you just move the 'private let lock = NSLock()' right to the top of the SessionDelegate class, you want see this anymore in Leaks. It's a weird thing in Swift.
I tried it but nothing changed.

The Request adds a block operation referencing self, to the TaskDelegate.queue. I can't remember what it is off-hand, something to do with setting a time maybe. The taskDelegate is itself a strong property of the Request. The taskDelegate queue is not resumed in two situations. The first is when SessionDelegate.taskDidComplete is wired up. The second is when taskDelegate.TaskDidCompleteWithError is wired up. I am not 100% sure of the second case, however the first I can see clearly in debug. The queue is never resumed and the whole request and task delegate leak due to a strong reference cycle held inside that queue. At least, that's what I think it is. I not the guru here. I just make the tea.
The last cleanup point is inside the completeTask block inside the SessionDelegate didCompleteWithError function I listed earlier. The code modification I made ensures that queue is resumed, the queue operation completes and the reference cycle is remove:
if let delegate = strongSelf[task]?.delegate {
delegate.queue.isSuspended = false
}
Request.resume() has this, but task is NOT nil so it does not get executed in this case...
guard let task = task else { delegate.queue.isSuspended = false ; return }
The only other place where isSuspended is set false is in TaskDelegate didCompleteWithError. That bit of code is never run if SessionDelegate.taskDidComplete is wired up. If taskDidComplete is not wired up and TaskDelegate.taskDidCompleteWithError is wired up instead, then again, the queue is never resumed and the request (and all its references) and the task delegate leak.
The separate issue of the NSLock() appearing in instruments is I believe, due to a bit of a mess with swift initialisation. It seems to occur in static instance classes. If you want something really odd, try adding a couple of string dictionaries to the the AppDelegate. I guess it has something to do with the way Instruments works but, silly as it sounds, repositioning properties makes a difference. It can of course, work both ways. Repositioning can make something else appear to leak. Its just something I discovered by accident.

AlamofireBackgroundSessionTest.zip
Here is the same test project with the SessionDelegate file modification in urlSession-didCompleteWithError, and the repositioned NSLock() just to make Instruments-leaks show pretty green lights.

@chamira, the NSLock() property has to be first - before any other property. Look what I did to SessionDelegate in the test project. I only moved this single property to the top. You have moved retrier, sessionManager and requests as well. Leave them were they were. It's a really silly issue. Laughable really.

@unusuallyreticent Thanks for the investigation, it helped narrow this down quite a bit. It turns out that, when the taskDidComplete closure is set, AF isn't calling it's own TaskDelegate, which means we lose all of the machinery in there that takes care of task completion. So not only would it leak but any response handlers attached would never be called. I've put up a tentative fix on the bug/1938 branch, pull it down with CocoaPods and see if it works for you. I still need to add some tests but it should fix the issues you're seeing.

@jshier - thanks. I'll try it out. And, sorry if I seemed a bit rude occasionally. It was unintentional - late nights, three projects on the go, too much coffee :-(
I was only looking at my particular corner case - using taskDidComplete so I didn't consider that TaskDelegate should still be called anyway. Makes sense. They aren't exclusive.
Did you try that silly trick with NSLock()? That has got to be the daftest trick. If anyone told me to do that I would consider them loony.

@jshier - that's solved it.

Hey everyone,
Sorry for being late to the party here. @jshier and I discussed the changes that he put up in the bug/1938 branch. I'm not surprised that fixed the issue for you @unusuallyreticent. You have to be very careful right now with the session override closures to make sure you don't shut off some of the functionality.
The change that @jshier made is switching that taskDidComplete closure over to a hook instead of an override. After quite a bit of debate, we realized that there shouldn't be any harm in this change since it was impossible to get the queue resumed without a huge amount of work. Therefore we're fairly confident this change won't break any clients out there now.
We're going to look to unify the override closures in AF5 or just remove them completely with the goal being to make sure their usage is clear and not easy to mess up. There is enough inconsistency right now that it can lead to issues like you experienced.
As for the NSLock leak, I've actually filed a radar against that specific leak and it was closed as a duplicate. That is not something we can do anything about right now unfortunately.
I've pushed up @jshier's bug fix into the master branch in d64afca3 and it will go out as part of the Alamofire 4.4.0 release here shortly.
Cheers. 🍻


Canonical and Hreflang Links output encoded

Environment

Operating System: Darwin
Node Version:     v18.17.1
Nuxt Version:     2.17.1
CLI Version:      3.10.0
Nitro Version:    -
Package Manager<EMAIL_ADDRESS>Builder:          webpack
User Config:      -
Runtime Modules:  -
Build Modules:    -

Reproduction
Version: "@nuxtjs/i18n": "^7.3.1"
Describe the bug
Canonical and Hreflang Links output encoded in the head.
Example:
<link data-n-head="ssr" data-hid="i18n-can" rel="canonical" href="/en/pages%2Ffaq">
According to our SEO team:
This issue affects the proper indexing of the pages, leading to SEO indexing problems and content duplication concerns.
The canonical link still serves the same content. The issue is that it's telling search engines to ignore the live link and focus on indexing the incorrect coded link that is orphaned.
I've a had look in<EMAIL_ADDRESS>Below is the output of  `this.$route':
fullPath: "/en/pages/faq"
path: "/en/pages/faq"
...

However, the links are generated with currentRoute which outputs below:
fullPath: "/en/pages%2Ffaq"
path: "/en/pages%2Ffaq"
...

Is there a reason that '/' characters are encoded, and is there a way it can be changed?
Additional context
No response
Logs
No response

Closing this as v7 is not being actively worked on (critical hotfixes only), if this still applies to v8 and later please open a new issue!


"Necessary topics are not subscribed yet" and the ego car doesn't move.

Hi, I followed the README, and repeated for several times. I have git-lfs installed, and it cloned the submodules.
How to get the ego car moving? Thank you!

@sparklin1999 same issue here. Have you found any solutions?

Hi, same issue here.

Same issue. Why was this closed?

Same issues

Me too

The not subscribed topics may not be an issue. Try to give the ego car a
goal in Rviz (just click goal in navi bar then click anywhere on the map)
and see if it moves.
On Wed, Jun 14, 2023 at 7:45 PM haibo @.***> wrote:

Me too
—
Reply to this email directly, view it on GitHub
https://github.com/carla-simulator/carla-autoware/issues/123#issuecomment-1591035291,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKTDLH7QAU2I745FXETTFODXLGP4XANCNFSM4Z3J63UQ
.
You are receiving this because you commented.Message ID:
@.***>


[SPARK-38041][SQL]DataFilter pushed down with PartitionFilter

Why are the changes needed?
At present, the Filter is divided into DataFilter and PartitionFilter when it is pushed down, but when the Filter removes the PartitionFilter, it means that all Partitions will scan all DataFilter conditions, which may cause full data scan.
This pr will push down dataFilter with PartitionFilter, and evaluate partitionFilter before push to parquet, which can push less filter to parquet.
Here is a example.
before
== Physical Plan ==
*(1) Filter (((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))
+- *(1) ColumnarToRow
   +- BatchScan[a#40L, b#41L, c#42, d#43] ParquetScan DataFilters: [(((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))], Format: parquet, Location: InMemoryFileIndex(1 paths)[path, PartitionFilters: [((c#42 = 0) OR ((c#42 >= 1) AND (c#42 < 3)))], PushedAggregation: [], PushedFilters: [Or(LessThan(a,10),GreaterThanOrEqual(a,10))], PushedGroupBy: [], ReadSchema: struct<a:bigint,b:bigint>, PushedFilters: [Or(LessThan(a,10),GreaterThanOrEqual(a,10))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

after
== Physical Plan ==
*(1) Filter (((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))
+- *(1) ColumnarToRow
   +- BatchScan[a#40L, b#41L, c#42, d#43] ParquetScan DataFilters: [(((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))], Format: parquet, Location: InMemoryFileIndex(1 paths)[path, PartitionFilters: [((c#42 = 0) OR ((c#42 >= 1) AND (c#42 < 3)))], PushedAggregation: [], PushedFilters: [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),Le..., PushedGroupBy: [], ReadSchema: struct<a:bigint,b:bigint>, PushedFilters: [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),LessThan(c,3)))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

Does this PR introduce any user-facing change?
No
How was this patch tested?
origin tests and add new test.

cc @huaxingao @HyukjinKwon @dongjoon-hyun @cloud-fan

@stczwd Thanks for working on this. I haven't looked at your implementation yet. Based on the PR description, this PR will push down both data filters and partition filters, but it seems to me that we only need to push down data filters, because the partition filters have already been handled by partition pruning.

cc @cloud-fan

@stczwd Thanks for working on this. I haven't looked at your implementation yet. Based on the PR description, this PR will push down both data filters and partition filters, but it seems to me that we only need to push down data filters, because the partition filters have already been handled by partition pruning.

Thanks for your attension. Yeap, partition filters have already been handled by partition pruning. However, the data filters, which pushed down, contains conditions for every partitions, which means we need scan all conditions for each partition.
Simple examples.

if condition is (a > 0 and c = 0) or c=2, then no datafilter will be pushed down, we will scan all data of partition(c=0) and partition(c=1);
if condition is (a > 10 and c=0) or (a <=10 and c=2), then the pushed filter will be a>10 or a<=10, we will also scan all data of partition(c=0) and partition(c=1);

@huaxingao I have changed the PR description，and listed the difference between the two plans, so that the difference can be better seen

Also, cc @sunchao and @MaxGekk.

This looks like an interesting optimization. For file sources, the pushed predicate can be dynamic: we can fill the value of the partition columns in the pushed predicate, when pushing down the predicate to Parquet at the task runtime.
cc @sigmod @maryannxue @viirya @sunchao @c21

Hmm, the first glance looks a bit confused, actually. Maybe it is good to improve the PR description. So, I guess this means, if the data filter is partition related, e.g. a < 10 is only for partition c = 0, and a >= 10 is for partitions c >= 1 and c < 3, then once we strip out partition filter during pushing down, we will end with a < 0 or a >= 10 which actually scans all data.
But we only want a < 10 in partition c = 0, and a >= 10 in other two partitions.
Is it correct?

+1 on what @viirya said - the PR description is a bit confusing, for instance I don't really know what column a and c are for when first looking at it. It'd be better if you can also put the original SQL query there.
Also I'm curious how Parquet handles pushed filters like (a < 10 and c == 0) or ((a >= 10 and c >= 1) and c < 3)), since the partition column are not materialized.

+1 to @sunchao and @viirya's points. Could we add the evidence of Parquet skipping files/row-groups (either a micro benchmark or some logs during execution or code pointers), when we push down partition filter here?

Hmm, the first glance looks a bit confused, actually. Maybe it is good to improve the PR description. So, I guess this means, if the data filter is partition related, e.g. a < 10 is only for partition c = 0, and a >= 10 is for partitions c >= 1 and c < 3, then once we strip out partition filter during pushing down, we will end with a < 0 or a >= 10 which actually scans all data.
But we only want a < 10 in partition c = 0, and a >= 10 in other two partitions.
Is it correct?

Yes, this is what I trying to do

+1 on what @viirya said - the PR description is a bit confusing, for instance I don't really know what column a and c are for when first looking at them. It'd be better if you can also put the original SQL query there.
Also I'm curious how Parquet handles pushed filters like (a < 10 and c == 0) or ((a >= 10 and c >= 1) and c < 3)), since the partition columns are not materialized.

Thanks for your attension, @sunchao @viirya @c21. Just as @cloud-fan said, we can push all filters, data filters and partition filters, to ParquetScan, and evaluate and prune partition filter with partition values at runtime, before pushed into parquet. With this, we can read a < 10 in partition c=0 and read a>=10 in partition c>=1 and c<2, instead of reading a<10 or a>=10 in partition c=0 and partition c>=1 and c<2.

@viirya @sunchao @c21 I have updated the PR description, does it look clear now?

.. and evaluate and prune partition filter with partition values at runtime, before pushed into parquet

Ah I see it now, thanks @stczwd .

Could we add the evidence of Parquet skipping files/row-groups (either a micro benchmark or some logs during execution or code pointers), when we push down partition filter here?

@c21 I have add some benchmark tests in FilterPushdownBenchmark, and run them in github actions. Test code can be found here.
Test result
OpenJDK 64-Bit Server VM 1.8.0_322-b06 on Linux 5.11.0-1028-azure
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
Data filter with partitions: ((a = 10 and part = 0) or (a = 10240 and part = 1) or (part = 2)):  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Parquet Vectorized with partition                                                                        3039           3157         122          5.2         193.2       1.0X
Parquet Vectorized with partition (Pushdown)                                                             1548           1568          15         10.2          98.4       2.0X

OpenJDK 64-Bit Server VM 1.8.0_322-b06 on Linux 5.11.0-1028-azure
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
Data filter with partitions: ((a > 10 and part = 0) or (a <= 10 and part >=1 and part < 3)):  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Parquet Vectorized with partition                                                                     2942           2997          40          5.3         187.1       1.0X
Parquet Vectorized with partition (Pushdown)                                                          1497           1513          15         10.5          95.2       2.0X

@stczwd Can you add the benchmark code to this pr and use GA to produce the benchmark results?

Also cc @dongjoon-hyun I'm not sure whether this optimization is also applicable to Orc file, and whether Orc also needs this optimization?

I think the optimization should be able to add for ORC as well.

I think the optimization should be able to add for ORC as well.

Great. I will create a new JIRA for ORC optimization, this PR will only support parquet optimization.

@stczwd The PR makes sense to me. I have one question: does your optimization work for V1 file source? Seems to me that data filters and partition filters are separated differently in V1 and V2 file sources and your optimization only work for V2 file source?

Seems to me that data filters and partition filters are separated differently in V1 and V2 file sources and your optimization only work for V2 file source?

Yes, this is only worked with v2 file source. V1 file source won't work even if spark.sql.parquet.filterPushdown.partition is set to true.

@stczwd Can you add the benchmark code to this pr and use GA to produce the benchmark results?

@LuciferYang The benchmark code and result generated by GA have been updated.

From the results of the new benchmark, it can show some performance improvements, also ping @wangyum

@stczwd
Thanks for the quick fix.
The optimization in your PR makes sense to me, but I think there is a problem. Use this filter (id > 0 AND p = 0) OR (id = 2 AND p = 1) as an example, the current physical plan is
*(1) Filter (((id#9L > 0) AND (p#10 = 0)) OR ((id#9L = 2) AND (p#10 = 1)))
+- *(1) ColumnarToRow
   +- BatchScan[id#9L, p#10] ParquetScan DataFilters: [(((id#9L > 0) AND (p#10 = 0)) OR ((id#9L = 2) AND (p#10 = 1)))], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/pt/_5f4sxy56x70dv9zpz032f0m0000gn/T/spark-51..., PartitionFilters: [((p#10 = 0) OR (p#10 = 1))], PushedAggregation: [], PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedGroupBy: [], ReadSchema: struct<id:bigint>, PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

This actually looks like a bug to me. I have fixed it to
== Physical Plan ==
*(1) Filter ((id#9L > 0) OR (id#9L = 2))
+- *(1) ColumnarToRow
   +- BatchScan[id#9L, p#10] ParquetScan DataFilters: [((id#9L > 0) OR (id#9L = 2))], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/pt/_5f4sxy56x70dv9zpz032f0m0000gn/T/spark-38..., PartitionFilters: [((p#10 = 0) OR (p#10 = 1))], PushedAggregation: [], PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedGroupBy: [], ReadSchema: struct<id:bigint>, PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

I will submit a PR for the fix.
I think your PR needs the data filters to be  [(((id#9L > 0) AND (p#10 = 0)) OR ((id#9L = 2) AND (p#10 = 1)))].
After my fix, you can still do your optimization, but need to change some code .

cc @cloud-fan for the above comment

Should (id > 0 AND p = 0) OR (id = 2 AND p = 1)  really be optimized to
== Physical Plan ==
*(1) Filter ((id#9L > 0) OR (id#9L = 2))
+- *(1) ColumnarToRow
   +- BatchScan[id#9L, p#10] ParquetScan DataFilters: [((id#9L > 0) OR (id#9L = 2))], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/pt/_5f4sxy56x70dv9zpz032f0m0000gn/T/spark-38..., PartitionFilters: [((p#10 = 0) OR (p#10 = 1))], PushedAggregation: [], PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedGroupBy: [], ReadSchema: struct<id:bigint>, PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

It looks wrong for record id=1, p=1, or did I get it wrong @huaxingao?

@LuciferYang
You are right. I got the post scan filter wrong. It should be
== Physical Plan ==
*(1) Filter (((id#9L > 0) AND (p#10 = 0)) OR ((id#9L = 2) AND (p#10 = 1)))
+- *(1) ColumnarToRow
   +- BatchScan[id#9L, p#10] ParquetScan DataFilters: [((id#9L > 0) OR (id#9L = 2))], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/pt/_5f4sxy56x70dv9zpz032f0m0000gn/T/spark-38..., PartitionFilters: [((p#10 = 0) OR (p#10 = 1))], PushedAggregation: [], PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedGroupBy: [], ReadSchema: struct<id:bigint>, PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

After my fix, you can still do your optimization, but need to change some code .

Hm, I think you mean the FileScanBuilder.pushFilters should split the partitionFilter and dataFilter? And for parquet or orc, they should overwrite the method and get the filters before split? is it what you mean? @huaxingao

Yes, I mean FileScanBuilder.pushFilters should split the partitionFilter and dataFilter, and only data filter should be pushed down to parquet/orc.  But not splitting doesn't cause any issue anyways because later on Spark checks the filter and only creates parquet/orc filters for data filters. I will leave this as is so  you can use it for your optimization.

@stczwd
What is the reason for your optimization not working with V1? is it because only data filters are pushed to the FileScan?
== Physical Plan ==
*(1) Filter (((id#9L > 0) AND (p#10 = 0)) OR ((id#9L = 2) AND (p#10 = 1)))
+- *(1) ColumnarToRow
   +- FileScan parquet [id#9L,p#10] Batched: true, DataFilters: [((id#9L > 0) OR (id#9L = 2))], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/pt/_5f4sxy56x70dv9zpz032f0m0000gn/T/spark-7d..., PartitionFilters: [((p#10 = 0) OR (p#10 = 1))], PushedFilters: [Or(GreaterThan(id,0),EqualTo(id,2))], ReadSchema: struct<id:bigint>

@stczwd What is the reason for your optimization not working with V1? is it because only data filters are pushed to the FileScan?

Yes, the data filters that pushed to the FileScan and the way filter pushed down are different in V1 and V2. Thus I thought start a work on V2 first and then optimize V1 based on the same ParquetFilters later.

How hard it is to support file source v1? This is a big PR and it's a pity if it only works for file source v2 which is off by default,

How hard it is to support file source v1? This is a big PR and it's a pity if it only works for file source v2 which is off by default,

@cloud-fan Thanks for your attension. Actually, there are not many changes in this PR, most of the changes are in the benchmark results.
As for supporting os file source v1,  we split dataFilters and partitionFilters at FileSourceStrategy.apply, and we need a way to push whole filters to Fileformat.buildReaderWithPartitionValues. Not a big PR, but it can be used in ORC v1 file source. Maybe another PR is better?

== Physical Plan ==
*(1) Filter (((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))
+- *(1) ColumnarToRow
   +- BatchScan[a#40L, b#41L, c#42, d#43] ParquetScan DataFilters: [(((a#40L < 10) AND (c#42 = 0)) OR (((a#40L >= 10) AND (c#42 >= 1)) AND (c#42 < 3)))], Format: parquet, Location: InMemoryFileIndex(1 paths)[path, PartitionFilters: [((c#42 = 0) OR ((c#42 >= 1) AND (c#42 < 3)))], PushedAggregation: [], PushedFilters: [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),Le..., PushedGroupBy: [], ReadSchema: struct<a:bigint,b:bigint>, PushedFilters: [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),LessThan(c,3)))], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []

Seems to me this plan might be a bit misleading. The data filters are actually [((id#9L > 0) OR (id#9L = 2))] and the predicate pushed down to Parquet is [((id#9L > 0) OR (id#9L = 2))]. With the pushed filter to be [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),LessThan(c,3)))], it makes me feel that we are constructing a Parquet predicate [Or(And(LessThan(a,10),EqualTo(c,0)),And(And(GreaterThanOrEqual(a,10),GreaterThanOrEqual(c,1)),LessThan(c,3)))] and push it down to Parquet.

The data filters are actually [((id#9L > 0) OR (id#9L = 2))] and the predicate pushed down to Parquet is [((id#9L > 0) OR (id#9L = 2))]

Hm, acutally, the predicate pushed down to Parquet should be [(id#9L > 0)] or [(id#9L = 2)], that's why we need runtime filter pruning.
BTW, it's the meaning of the pushed filter changed. In this PR, the pushed filter means the filter of push to runtime, not the filter of push to parquet. Maybe using PushedRuntimeFilters instande of PushedFilters?

Maybe using PushedRuntimeFilters instande of PushedFilters

Sounds OK to me.
The property name is a bit confusing too. Maybe change it as well?

Sounds OK to me. The property name is a bit confusing too. Maybe change it as well?

Not get what property name means. You mean this should be also changed in ParquetScan.description and ParquetScan.getMetaData?

Sorry I didn't make it clear. I feel PARQUET_FILTER_PUSHDOWN_PARTITION_ENABLED is a bit misleading. We are not really push down partition filters. What we do is to dynamically push down data filters based on the partition values. Shall we call this PARQUET_FILTER_RUNTIME_PUSHDOWN_ENABLED or PARQUET_FILTER_DYNAMICALLY_PUSHDOWN_ENABLED?

Shall we call this PARQUET_FILTER_RUNTIME_PUSHDOWN_ENABLED or PARQUET_FILTER_DYNAMICALLY_PUSHDOWN_ENABLED?

Ah, I got it. The second one sounds good to me, I will change it too.

cc @huaxingao and @cloud-fan

@huaxingao any more question about this pr? can this pr be merged now?

@jackylee-ch Sorry I didn't have time to take a look yet. Will try to take a look during the weekend. It's probably a little rush to merge in 3.3.0.

@jackylee-ch Sorry I didn't have time to take a look yet. Will try to take a look during the weekend. It's probably a little rush to merge in 3.3.0.

Okey, I'm just asking. It's ok to me that merging this pr to 3.4.0, I will change the affect versions.

@cloud-fan
I think we probably need to reach a consensus here :)
before this patch, the physical plan has
Filter:
(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))

ParquetScan DataFilters:
[(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))]

PartitionFilters:
[((part#15 = 0) OR (part#15 = 1))]

PushedFilters:
[Or(LessThan(id,2),GreaterThan(id,3))]

after this patch, the physical plan has
Filter:
(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))

ParquetScan DataFilters:
[(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))]

PartitionFilters:
[((part#15 = 0) OR (part#15 = 1))]

PushedDynamicalFilters:
[Or(And(LessThan(id,2),EqualTo(part,0)),And(GreaterThan(id,3),EqualTo(part,1)))]

Please note that PushedFilters is changed to PushedDynamicalFilters. [Or(And(LessThan(id,2),EqualTo(part,0)),And(GreaterThan(id,3),EqualTo(part,1)))] means data filter id < 2 is dynamically pushed to partition 0 and data filter id > 3 is dynamically pushed to partition 1.
Does this look OK to you?

At least I think this is OK. Only pushing down the filter belonging to the current partition is conducive to reducing the file scan.

@cloud-fan I think we probably need to reach a consensus here :)
before this patch, the physical plan has
Filter:
(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))

ParquetScan DataFilters:
[(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))]

PartitionFilters:
[((part#15 = 0) OR (part#15 = 1))]

PushedFilters:
[Or(LessThan(id,2),GreaterThan(id,3))]

after this patch, the physical plan has
Filter:
(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))

ParquetScan DataFilters:
[(((id#14 < 2) AND (part#15 = 0)) OR ((id#14 > 3) AND (part#15 = 1)))]

PartitionFilters:
[((part#15 = 0) OR (part#15 = 1))]

PushedDynamicalFilters:
[Or(And(LessThan(id,2),EqualTo(part,0)),And(GreaterThan(id,3),EqualTo(part,1)))]

Please note that PushedFilters is changed to PushedDynamicalFilters. [Or(And(LessThan(id,2),EqualTo(part,0)),And(GreaterThan(id,3),EqualTo(part,1)))] means data filter id < 2 is dynamically pushed to partition 0 and data filter id > 3 is dynamically pushed to partition 1.
Does this look OK to you?

Kindly remind @cloud-fan

cc @huaxingao @cloud-fan. This PR has been ready for a long time now, can we merge this PR?
Also cc @sunchao @viirya

@jackylee-ch Thanks for your patience. The approach looks good to me. I concern a bit about the logging, though.
PushedDynamicalFilters:
[Or(And(LessThan(id,2),EqualTo(part,0)),And(GreaterThan(id,3),EqualTo(part,1)))]

It's not very clear, but I don't have an easy way to make it better.
Since this PR has quite some changes, I prefer to get some opinions from others.

@huaxingao Maybe remove the Stale tag?
Also cc @cloud-fan @sunchao


Introduce lima in macOS

Resolves GH-146
Closes GH-911 for now
Closes GH-909 for now
Add comment about GH-910

lima team looks addressing to release 1.0.0 these days. So I may add config.yml, however I would like to pend until 1.0.0


Non intuitive select "all_of" name

Oks, this is confusing, the meaning of "select" is about the checks, which vars should be checked.
Second, "all_of" is about that "must only exists columns from here".
"all_of" and "select" are not intuitive at all.
Data.frame(columns = list(a = typed::Integer()), select = "all_of") ? a
#This should be valid
a <- data.frame()

fixed by https://github.com/CIT-UAI/geotypes/pull/9


Add Option to Specify Other LLM Providers in .env

Is your feature request related to a problem? Please describe.
The current implementation supports only Azure OpenAI as the LLM provider. It lacks the flexibility to support other LLM providers such as OpenAI directly, Anthropic, Google or Groq. This would open opportunities for easy future migration to more efficient provider.
Describe the solution you'd like

Update the .env configuration file to support specifying alternative LLM providers.
Modify the backend in main.py to read and initialize the appropriate LLM client based on the selected provider from the .env file.
Implement helper functions to set up clients for the new LLM providers.

Additional context

Update .env to include environment variables for configuring alternative LLM providers.
Thoroughly test switching between different LLM providers to ensure smooth initialization and operation.

/bounty $20

/attempt #10
Hey there! 👋 Here's my plan for adding multi-LLM support to the project:
Adding Multiple LLM Providers - Implementation Plan
What will i do:
Create a simple but flexible system to swap between different AI providers (OpenAI, Anthropic, Groq, etc.)
Update our config to handle different provider settings
Refactor the existing Azure OpenAI code to use this new system
Add proper error handling for when providers act up
Main changes needed:
New provider system managing diff llms
Update to our .env setup
Tweaks to the database to track which provider was used
Some UI updates to show/select providers
What do you think? doe this seems right?

@madman024 sounds high level but sure you can try


Forgive me, I am very foolish. I don't know how to correctly include header files in the VS 2022 project to prevent SDK headers from reporting errors

Hey, we are not offering any support on how to use the SDKs our software generates


[DOCS] add readme badges

Description
I want to add readme badges.Please assign this issue to me
Screenshots

🥦 Browser
Brave
Checklist ✅

[X] I checked and didn't find similar issue
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (blank for no)

Sounds cool -  we can add the Opensource, and after that maybe just these 3 but all in one line :


Beam services operator does not restart failed flink jobs properly

After the core-services job fails due to some unknown reason, the CRD goes to 'UNKNOWN' state and the kopf operator is unable to  react on this. Furthermore, if the job is deleted in the unknown state, the operator gets stuck deleting a job that does not exist in flink anymore. The manual solution I have found is to delete the CRD, and the beam services operator, then rollback the CRD with helm.

Official solution is to let Flink restart it. CRD is to deploy and update the job but cannot restart a job because Flink needs also to rebuild the state of the job.
Today, Flink is configured to try 10 restarts and then fail. We will move the the Exponential-restart mode where Flink tries indefinitely to restart the job but adds some backoff time when it fails.

Issue is recently solved, closing...


CLI output is noisy

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2096
Originally created by @DouweM on 2020-06-15 21:27:04

$ meltano config tap-gitlab
[2020-06-15 16:19:31,313|I|MainThread|alembic.runtime.migration] Context impl SQLiteImpl.
[2020-06-15 16:19:31,313|I|MainThread|alembic.runtime.migration] Will assume non-transactional DDL.
System database up-to-date.
{"api_url": "https://gitlab.com", "private_token": "", "groups": "meltano", "projects": "meltano/demo-project", "ultimate_license": false, "start_date": "2020-05-01"}

The first 3 lines of output are a distraction more than anything.

View 2 previous comments from the original issue on GitLab


[Bug]: vibe.exe cannot write to Documents folder (ransomware protection enabled)

What happened?
I run into a strange bug.
The very first test I run when recording through the mic worked, then nothing. The wav files were not written and I saw that write access was blocked by Windows Defender built-in ransomware protection.  I had to explicitly allow vibe.exe, and then it started working again.
Suggestion: write temp files to the application's AppData folder, not Documents. Writing to AppData is, as far as I know, what MS recommends as best practice.
Steps to reproduce

step one install, enable ransomware protection
step two, record a message
3.. step three, try recording another message -
check ms security center logs to the access denied.

What OS are you seeing the problem on?
Window
Relevant log output
No response

I enabled fallback to write into temp folder. do you need the audio file?


Possible Improvements with retrieving list values

I noticed in your script whenever a scan query finds a list type, the length of the list is grabbed, and the values are pulled index by index. https://github.com/fritzy/node-redisscan/blob/master/index.js#L57
There is a redis command LRANGE which can pull a range of values from the list, and would reduce the queries submitted to redis if the LRANGE were to pull index's 0 to -1 (all records) or possibly split the length out into sets of 10, 20, etc and pass an array into the callback, or split the array and pass each value to the callback to maintain current syntax.
I'm curious to know if this was an oversight, or there is a reason behind using LINDEX over LRANGE. Only concern behind LRANGE I can imagen is if the list is HUGE in redis, a reply may never be received due to the mass amounts of data to process, causing the socket to timeout, which splitting into sets would avoid.
A fork will be made.

Your fork has proved to be much more stable when scanning large amounts of values (500+). The current command often leads to objects being "skipped" when working with large value sets that are updated often.


Outbound TCP Idle timout is now possible

Hi Team,
I know you've mentioned in the notes at the start of the Document that for new methods are present for outbound connectivity and flows. But in the below document i can find there are outdated topic still present which lead to confusion.
in the section "Modify the application to use less aggressive retry logic"
You've mentioned "Ephemeral ports have a 4-minute idle timeout (not adjustable)." which can be changed now with standard LB.
same thing in section "Ephemeral ports have a 4-minute idle timeout (not adjustable)."
you've mentioned about "Outbound connections have a 4-minute idle timeout. This timeout is not adjustable"
which can be adjustable using standard LB.
Can you please mention about standard LB and configurable limits in both the section. It'll be more clear.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a96406ce-2ffc-cdb3-1234-5154ccd5cc76
Version Independent ID: f840a092-e1bf-16e2-1916-4270cdfbbcdd
Content: Outbound connections in Azure
Content Source: articles/load-balancer/load-balancer-outbound-connections.md
Service: load-balancer
GitHub Login: @asudbring
Microsoft Alias: allensu

@aquibqureshi26 , We will investigate and get back to you shortly.

any update?

Hi Karthik,
Yes you are right, My question was something else. I think this configurable limit should also be mentioned in the article lines which I've highlighted while raising this thread.

@aquibqureshi26 , It is mentioned under Outbound rules page.

Sure Karthik i know it is updated on the other doc. But on this doc it is not updated hence i thought it is little misleading.

Sending over to PM for review.  It looks like this article needs a freshness review to have Standard Load balancer information added to the existing information.
#assign:@anavinahar

This has been fixed.
#please-close


Please add me to the Kivy Garden

Link for a merged garden PR or a proposed new flower: https://github.com/kivy-garden/garden.resizable_behavior/commit/d9b9bf1fdb222b6d6f344e23e2dd7777c66e4d43

I don't understand? This is already in garden.


feat(ui) : test portfolio roadmap

Checklist:

[X] I have read and followed the contribution guidelines.
[X] I have read and followed the how to open a pull request guide.
[X] My pull request targets the main branch of freeCodeCamp.
[X] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Follow up PR to #53963 where the icons are analyzed. For some reason the certified user didn't register as complete for the english curriculum and I'm a little confused.

Yea, the certified user probably doesn't have the English challenges completed. Also, there's no certification for the English curriculum yet.

Yea, the certified user probably doesn't have the English challenges completed. Also, there's no certification for the English curriculum yet.

Okay I'll go ahead and alter my tests to not include the English curriculum for the certified user.

This does not have to be a e2e test because it is testing a simple non-interactive functionality. But this should do that job. I left some comments for your review.

Okay now I've got it.

The test is failing because the signed in user for whatever reason doesn't have those certs and I'm not sure how to fix it so the test user has the right data.

I will take a look @a2937!

Hey @a2937,
Unfortunately I have to close this because our certified user is not up to date. This means that the icons will never show up because most certifications are not completely finished.
Feel free to reopen this when the certified user is back up to date.
Sorry of the inconvenience


draggable component inside a parent  in responsive screens

I have been testing the draggable component in my Chrome, and it works in a normal webpage. But when I change the display into the responsive screen, the drag doesn't work when <vue2-interact-draggable> is under a parent
 class:
So, the following works
<template>
    <vue2-interact-draggable>

    </vue2-interact-draggablet>
</template>

But the below doesn't work:
<template>

     <vue2-interact-draggable>

    </vue2-interact-draggablet>

</template>

I fixed it by adding following in the div css:
-ms-touch-action: none;
touch-action: none;


convert to typescript

Description
Converts the backend codebase to typescript
Fixes/resolves #90
Type of change

[x] Improvement (non-breaking change which improves existing functionality)

Checklist:
Leave blank if not applicable
I have completed these steps when making this pull request:

[x] I have checked that the PR is from a forked repository
[x] I have assigned my name to the issue
[x] I have moved the issue to the In Progress column
[x] I have labelled the PR appropriately
[x] I have assigned people responsible to the PR

Before opening the PR for review:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have documented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[x] I have moved the linked issue to the Review in Progress column

the tests are very very broken it's not just you


Self-service community WDC listing

Proposal!  Let's make listing WDCs in the community section self-service.  This PR takes inspiration from yeoman and the way it uses the npm registry to make app generators discoverable and applies it to Tableau Web Data Connectors.
What, how?
It's a small angular app built on top of npmsearch.com; results are filtered down to just npm packages containing the keyword tableau-wdc and whose names end with -wdc.
Looks like this in action.
For Review

Code Updates, excluding libraries/dependencies.
Documentation Updates

Ping @jdomingu @lbrendanl @benlower

Looks really sharp! Nice. I think there might be a bit of hesitancy around changing the submission process while the public hackathon is going on this month, but we should definitely dig in and review. @lbrendanl @sdesmond46

@lbrendanl @benlower See https://github.com/tableau/webdataconnector/pull/39/commits/5f087c7361e44d7aa34504cecc469df2d2199426 for diffs based on our discussion today.


[Workspace] Elimiate creating symlinks in Packages/

This is no longer needed.

Are you sure we shouldn't keep this around for a bit as we transition over, in case users have other things expecting the packages in those locations? It would allow us to differentiate between a dependency on that and other problems with the new implementation.

I can email swift-build-dev asking if anyone is using this directory directly, otherwise I don't think we should keep it unless needed. Also, edit and this can't coexist unless we want to chose some other directory for editable packages

Ok, I'm fine ditching it... esp since it seems like this makes sense for editable packages.

@swift-ci please test


views: add 'allOf' support

Signed-off-by: Pamfilos Fokianos<EMAIL_ADDRESS>

@pamfilos Can you make the tests pass.
@nharraud Can you have a  look at this one.

Sorry for the delay
@pamfilos First thank you for this PR.
Zenodo uses another approach which is precompiling the JSON Schemas with a custom CLI command
What about doing something similar?
This solution should be more efficient as it is done once and offline.
Otherwise we can try to do it here but the function needs to be more generic (multiple allOf can be anywhere in the JSON Schema, not just one at the root), and it would need a few tests so that it doesn't get broken later.

Closing as superseded by #54


Scroll and rocket image

Is this the expected behaviour?

No, the backdrop must have a specific height and must not overflow.


Fix typo in load-image.js

Fixed typo so that reject on Line 14 passes the correct 'message' variable, instead of non-existent 'msg' variable.

Hi @mpj! Made a quick typo fix so that this code works. Thanks for everything!


feat: add faucet & mint on bottom-left corner

✨ Fix #226

[x] This pull request also changes some texts on create wallet page

Need to fix auto faucet before be able to review


Safe fix for #13015

Description:
Likely a safer version of #13023.
Reinstate old code in bootstrap.async_from_config_dict
Related issue (if applicable): fixes #13015

Reinstate https://github.com/home-assistant/home-assistant/commit/6734c966b33a23c5d72b42661048c2f245c47882#diff-9c7df5b7528228f37997fad85daa65caL122
But probably #13023 should be merged into dev as final fix


Fix NULL values in "covered distance"

Check the python script code to verify if after validation some nan values were still in the datafram. dataframe.isnull().sum() gave 0 for each column of the database.
Check if there was effectively some null values in the database with a query and nothing resulted. To be sure of the validity of the query, I've tried to change a value of a cell in null to see if it was filtering or not and it was working

Results: It was just typeorm requiring for setting the column as not nullable (even if only a read-only method was implemented). Setting all the types and costraints solves the problem


Add batch upload functionality

What does this PR do

Adds batch upload functionality
Adds progress bar status
Provides csv download capability

How should this PR be tested?
flutter run
Checklist before requesting a review

[ ] I have performed a self-review of my code

The wallet balance doesn't seem to decrement after the app is run - are we intending to fix that as a part of this PR too?


Bug fixed: error on default timezone not set

date_default_timezone_set('UTC') should be moved few lines above as the DateTime() is called previously.


When running a test suite that uses TestWidgetsFlutterBinding, all HTTP requests will return status code 400, and no network request will actually be made.

When you try to make http request in your test file, you have to mock HttpClient and when you mock HttpClient every time you make a request it always will return statusCode 400. I dont know why it is hardcoded to response every time with statusCied 400. This should not happen. We should have access to write what status code we want. This is almost same with this #35318 but with a difference that i am talking about Unit testing and not Widget testing.
I am forced to use TestWidgetsFlutterBinding.ensureInitialized() immediately after main().
I am not posting code to reproduce but if you need it feel free to ask.
Flutter 1.22.6 • channel stable
This is the class that affects my tests if you search statusCode:
https://chromium.googlesource.com/external/github.com/flutter/flutter/+/refs/tags/v0.5.1/packages/flutter_test/lib/src/binding.dart
https://api.flutter.dev/flutter/flutter_test/TestWidgetsFlutterBinding-class.html

Hi @kristimeculi
Unless you provide a reproducible code sample I will assume this is similar to #35318 and will probably close it. Please provide a reproducible code sample.

I am forced to use TestWidgetsFlutterBinding.ensureInitialized() immediately after main().

Just a question, why you need this on unit tests?
Thank you

`
import 'package:bloc_test/bloc_test.dart';
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';
void main() {
TestWidgetsFlutterBinding.ensureInitialized();
MethodChannel channelOne;
MethodChannel channelTwo;
MyBloc myBloc;
setUp(() {
channelOne = MethodChannel("channelOne");
channelTwo = MethodChannel('channelTwo');

});
tearDown(() {
myBloc.close();
channelOne.setMockMethodCallHandler(null);
channelTwo.setMockMethodCallHandler(null);
});
  blocTest(
    "Test if Grill credentials are right and connection is true",
    build: () async {
      channelOne.setMockMethodCallHandler((MethodCall methodCall) async {
        if (methodCall.method == "channelOne") {
          return true;
        }
      });
      channelTwo.setMockMethodCallHandler((MethodCall methodCall) async {
        if (methodCall.method == "channelTwo") {
          return "some text";
        }
      });

      return myBloc;
    },
    act: (bloc) => bloc.add(
      MyBlocEvent(),
    ),
    expect: [
      FirstState(),
      SecondState(),
    ],
  );

}`
Warning: At least one test in this suite creates an HttpClient. When
running a test suite that uses TestWidgetsFlutterBinding, all HTTP
requests will return status code 400, and no network request will
actually be made. Any test expecting a real network connection and
status code will fail.
To test code that needs an HttpClient, provide your own HttpClient
implementation to the code under test, so that your test can
consistently provide a testable response to the code under test.

Hi @kristimeculi
Unless you provide a reproducible code sample I will assume this is similar to #35318 and will probably close it. Please provide a reproducible code sample.

I am forced to use TestWidgetsFlutterBinding.ensureInitialized() immediately after main().

Just a question, why you need this on unit tests?
Thank you

Because it gives me this error when i dont use it.
ServicesBinding.defaultBinaryMessenger was accessed before the binding was initialized.
If you're running an application and need to access the binary messenger before runApp() has been called (for example, during plugin initialization), then you need to explicitly call the WidgetsFlutterBinding.ensureInitialized() first.
If you're running a test, you can call the TestWidgetsFlutterBinding.ensureInitialized() as the first line in your test's main() method to initialize the binding.

Hi @kristimeculi
Can you please provide a code that does not use any third-party plugin/package. The code above seems to use a bloc package.
Thank you

Hi @pedromassangocode same thing happen again and with this test
test('Test 2', () {
final expectedResponse = [
FirstState(),
SecondState(),
];
channelOne.setMockMethodCallHandler((MethodCall methodCall) async {
  if (methodCall.method == "channelOne") {
    return true;
  }
});
channelTwo.setMockMethodCallHandler((MethodCall methodCall) async {
  if (methodCall.method == "channelTwo") {
    return "some text";
  }
});

expectLater(
  myBloc,
  emitsInOrder(expectedResponse),
);

myBloc.add(MyBlocEvent());

});

Unit tests are meant to be reproducible and self-contained (hence the word unit), by interacting with the network you violate that principle. Overrides like this are on by default during non-integration tests to make sure you don't depend on anything external by accident.
I'd highly recommend doing integration tests instead, which are better suited for those kinds of tasks.
Otherwise, you should be able to override the test binding to turn off overrideHttpClient:
class CustomBindings extends AutomatedTestWidgetsFlutterBinding {
  @override
  bool get overrideHttpClient => false;
}

void main() {
  CustomBindings();

  testWidgets('test', (tester) async {
    // ...
  }
}

Hi @kristimeculi
Please see the comment above for suggestions.

Hi @kristimeculi
Please see the comment above for suggestions.

I have tried it and it does not work
I am not talking about Widget testing

I have tried it and it does not work

Please provide a test case that does not use any third-party package/plugin.

2. I am not talking about Widget testing

@PixelToast  gave a suggestion about that in https://github.com/flutter/flutter/issues/77245#issuecomment-791149778.

void main() {
//TestWidgetsFlutterBinding.ensureInitialized();
const MethodChannel channel = MethodChannel('plugins.flutter.io/connectivity');
final List log = [];
var something = Something("https://google.com");
setUp(() {
channel.setMockMethodCallHandler((MethodCall methodCall) async {
log.add(methodCall);
return '';
});
log.clear();

});
test('retrieveLostData get error response', () async {
channel.setMockMethodCallHandler((MethodCall methodCall) async {
return true;
});
//WifiHelper().sendRouterCredentialsWithHTTP(something) return true if http request return stausCode 200
final  response = await WifiHelper().sendRouterCredentialsWithHTTP(something); //
expect(response, true);
});
}
Error :
**package:flutter/src/services/binary_messenger.dart 92:7    defaultBinaryMessenger.
package:flutter/src/services/binary_messenger.dart 105:4   defaultBinaryMessenger
package:flutter/src/services/platform_channel.dart 143:62  MethodChannel.binaryMessenger
package:flutter/src/services/platform_channel.dart 414:5   MethodChannel.setMockMethodCallHandler
test/sample_test.dart 18:13                                main.
ServicesBinding.defaultBinaryMessenger was accessed before the binding was initialized.
If you're running an application and need to access the binary messenger before runApp() has been called (for example, during plugin initialization), then you need to explicitly call the WidgetsFlutterBinding.ensureInitialized() first.
If you're running a test, you can call the TestWidgetsFlutterBinding.ensureInitialized() as the first line in your test's main() method to initialize the binding.**
If I use TestWidgetsFlutterBinding.ensureInitialized() it return this Error:
Expected:
Actual:
Warning: At least one test in this suite creates an HttpClient. When
running a test suite that uses TestWidgetsFlutterBinding, all HTTP
requests will return status code 400, and no network request will
actually be made. Any test expecting a real network connection and
status code will fail.
To test code that needs an HttpClient, provide your own HttpClient
implementation to the code under test, so that your test can
consistently provide a testable response to the code under test.

Hi @kristimeculi
Your code is not runnable as it contains references for files I don't own.

Also I don't see you to override the test binding as suggested in https://github.com/flutter/flutter/issues/77245#issuecomment-791149778, as you can see your warning is suggesting the same.

Can you show me a full example how to use it.

Hi @kristimeculi
If you are looking for help I would suggest you to close this issue and look for the related documentation or ask on some of the Flutter community (see https://flutter.dev/community) as we don't use GitHub issues to provide assistance.
There is an example in this thread, see https://github.com/flutter/flutter/issues/77245#issuecomment-791149778.
Thank you

I have test @PixelToast  method and it makes real http request thats  not what i was looking for. I want to override statusCode in _MockHttpResponse that is used by _MockHttpClient.

@kristimeculi without a valid reproducible code and clear explanation, I think there is nothing the Flutter team can do about this issue.

For what do you need the code because i have a lot of classes that are connected with each other and i cannot share my project here. You do not need reproducible code for this i am telling you what is the problem.
The problem is because i can not test my app because of this _MockHttpResponse. We need to override statusCode not only to use statusCode => 400 as it is in flutter/packages/flutter_test/lib/src/_binding_io.dart class. That's the issue we need something to override this or something to workaround.

That's the issue we need something to override this or something to workaround.

This was already provided in https://github.com/flutter/flutter/issues/77245#issuecomment-791149778.

Reading back on our conversation on Discord I think I understand the question, @kristimeculi is trying to mock http requests and is confused by the fact that the native binding overrides the default implementation.
To inject a mocked http client in a test you can either:
a) Wrap main in HttpOverrides to override the creation of dart:io HttpClients (what the test binding does).
b) Instead of constructing http clients directly, let the test pass a factory to your widget / state management layer.
The second option seems simpler if you use package:http:
http.Client defaultCreateClient() => http.Client();

// Call from your state management layer to create a new client
http.Client Function() createClient = defaultCreateClient;

http.Client mockCreateClient() => MockClient((request) async {
  // ...
});

void main() {
  createClient = mockCreateClient;
  testWidgets('test', (tester) async {
    // ...
  }
}

i have fixed with the code in file i know it is not the best way but for the moment since flutter team does not do anything about this problem that's a workaround.
override.txt

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.


[ZEPPELIN-2216] general solution to precode. refactoring jdbc precode

What is this PR for?
General solution to execute precode. Refactoring jdbc precode using general solution. Task contains to subtasks: executeAfterOpen, executeBeforeClose. executeBeforeClose not done because we need the context so there is a solution only for executeAfterOpen.
What type of PR is it?
Feature | Refactoring
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2216
How should this be tested?

Add parameter zeppelin.PySparkInterpreter.precode someVar='text'
Execute

%pyspark
print(someVar)

Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

ready to review

prevous discussion #2096

@Leemoonsoo
prevous discussion #2096

Let me take a look in this weekend!

LGTM and merge to master if no further discussions.


⚠ Bofrost.be has degraded performance

In edb5b4e, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1403 ms

Resolved: Bofrost.be performance has improved in d920a85 after 8 minutes.


PHOENIX-5980 Test for MUTATION_BATCH_FAILED_SIZE and DELETE_BATCH_FAILED_SIZE metrics

@shahrs87 Please review when you have time.

Thanks @shahrs87 . Finally i was able to reproduce the bug for transactional table. Have not looked into the solution as of now but updated the test to repro the issue. 😄

@shahrs87 Looks like there a reason that we only return 1 for each batch for transactional tables. When it is transactional tables they consider it as one mutation as a whole. But if we really wanted to get number of deletes that were part of batch we need to fetch it using
DELETE_BATCH_FAILED_SIZE
instead of
MUTATION_BATCH_FAILED_SIZE

On another hand it looks like it is working ine because in transaction table one batch is one transaction so one transaction(think mutation) failed but if we want to know number of delete operations within that mutatiion we can check through
DELETE_BATCH_FAILED_SIZE

Thanks for the review @tkhurana and @virajjasani . Have updated the PR with your suggestions where applicable.

Looks good to go. @tkhurana anything from your side?

@mnpoonia one minor comment. Other than that looks good

@tkhurana Done

@mnpoonia Just to make sure my understanding is right, the current behavior is correct ?

When it is transactional tables they consider it as one mutation as a whole. But if we really wanted to get number of deletes that were part of batch we need to fetch it using
DELETE_BATCH_FAILED_SIZE
instead of
MUTATION_BATCH_FAILED_SIZE

I think we need to use DELETE_BATCH_FAILED_SIZE, right? That was the reason this JIRA was created in the first place. Am I understanding something wrong? Table being transactional doesn't have to do anything with how many deletes failed in that batch. Both are orthogonal. We tried to delete 4 rows and that failed. So the metric should reflect how many rows we tried to delete and failed.

We should fix the logic for deletes/mixed delete + upsert mutation batch failures.

The above line from the JIRA description. I don't understand what the author meant by mixed delete + upsert mutation batch failures. Does he mean if we perform delete and upsert mutation in 1 transaction, it will update the metric just by 1 also? Also what is the behavior if UPSERT SELECT queries are run?
@mnpoonia  @virajjasani @tkhurana

I think we need to use DELETE_BATCH_FAILED_SIZE, right? That was the reason this JIRA was created in the first place. Am I understanding something wrong? Table being transactional doesn't have to do anything with how many deletes failed in that batch. Both are orthogonal. We tried to delete 4 rows and that failed. So the metric should reflect how many rows we tried to delete and failed.

@shahrs87 You are right. table being transactional or not doesn't change the value of DELETE_BATCH_FAILED_SIZE. But when it  comes to  mutation  while we are in  transactional table we consider a set of command as  single mutation and that can be only reason i consider that MUTATION_BATCH_FAILED_SIZE differs.
So in that sense it looks fine. What do you think we are missing here?

I am bit confused. Chinmay's description on the jira did not have any mention of transactional usecase.
In the conn.commit() path, we get the number of mutations that failed to be committed in the catch block of MutationState.sendMutations() (see here).

In the jira description, he added link to the above code.He referred to getUncommittedStatementIndexes method which internally updates uncommittedStatementIndexes array. The only place where uncommittedStatementIndexes is being updated is addUncommittedStatementIndexes which is called from MutationState#send method iff table is transactional. See code here. So I assumed that this jira is for transactional table only. @virajjasani

But when it comes to mutation while we are in transactional table we consider a set of command as single mutation and that can be only reason i consider that MUTATION_BATCH_FAILED_SIZE differs.

When you look at the metrics independently, it will look like only 1 mutation failed in the batch even if 100 mutations have failed. We don't have different metrics for transactional table and non-transactional table and we shouldn't also. If a table is transactional we want to revert the state to original state if any of the mutations failed but why we want to treat metrics differently? @mnpoonia

@shahrs87 I think now i understand what you are retrying to say. Now that i look at the metric name again i think we should not consider anything special about this metric if table is tx or non tx. If we ever need special cases we can create special metric with some prefix or suffix indicating it is for tx tables.
Also  if  we look at the code again  ideally a delete statement should not be part of a batch else(we can talk about it some other day). The problem with deletes combing with upsert of having multiple  delete in same batch is as below

If we have  a batch with two delete statements and if one statement failed we will get a number saying that x num of deletes failed. But we don't know if it was first delete stmt or second or both failed. Maybe there is a smart way to figure out which i will look in code.

@shahrs87 i have tried solving the discrepancy. need your review to see if this makes sense
@tkhurana @virajjasani Nice if you could also skim through and provide feedback.

@virajjasani FYI.  Updated code with suggestions

@shahrs87 @tkhurana do you have bandwidth to take a look again?

Thanks @virajjasani


Fixed validate-vendor.bash to work with current dep implementation.

Fixes #645
I added sed command to validate-vendor.bash to strip memo from Gopkg.lock
It works for both GNU sed (Linux) and BSD sed (MacOS).
Here failed test build before running dep ensure: https://travis-ci.org/golang/dep/builds/235966915
And success build after running dep ensure: https://travis-ci.org/golang/dep/builds/235984505

oh awesome, thank you!

Thanks for review.
Just one not connected question.
Did reviewers receive notification about new changes after they requested change?
Or I need to manually add comment and ask for next review round?
For example, have you received notification after I pushed new changes to https://github.com/golang/dep/pull/635 ?
(I'am not pushing you to review just want to learn about workflow)
Thanks.

no worries - yep, i'm notified when you push changes or comment.


Implement NgbDatepicker to Formly

Question
A clear and concise description of what the question is.

I want to add ngbootstrap date picker as a type to formly like the image below. Can anyone show me the road I should follow?

define a custom field type https://formly.dev/guide/custom-formly-field


Update broken link to coverage.py docs

Spotted while looking into #203. I see other parts of the documentation (https://coveralls-python.readthedocs.io/en/latest/tips/coveragerc.html) already link to the latest version, hence why I propose to link to it here as well.

Ah, good catch, thanks!


[CodeComplete] Workaround a bug that parameterized protocols without 'any' cannot be mangled

typealias is currently allowed to refer to a protocol without the any keyword. This breaks mangling the typealias type into a USR will crash because parameterized protocols are expected to be any types.
Implement a SourceKit-specific minimal workaround for that problem by not computing USRBasedTypes for parameterized protocols.
rdar://98623438

@swift-ci Please smoke test

OK, I’ve decided to use this PR to implement a minimal workaround to avoid the crash on the SourceKit side so the underlying issue can be fixed independently.

@swift-ci Please smoke test

@swift-ci Please smoke test


Fix codecov paths

Codecov does not pick up on all files, e.g. aioredis/connection.py located at this commit in codecov: https://codecov.io/gh/aio-libs/aioredis-py/tree/b9694797d452bb8d4ca26d2dc14ce73068575130. Resolve via comment in codecov forum: https://community.codecov.io/t/missing-files-in-codecov-but-not-in-report/825/9

Ok well that didn't work

Boom e6d759d got it working

Thanks for this!!

Not yet @seandstewart remove the ./ first

all squash and merge should make it into one commit instead of 3

also let's see if that commit also broke it... hopefully not :)

Still missing init.py for some reason... though it's just imports so I think we're gtg

curiously on master branch I can't view the files no more... it says 404 not found... ffs no wonder the file structure was all in a line. So although the coverage worked, codecov couldn't understand relative urls perhaps?


DBZ-7376 Fix logging for schema only recovery in mysql

https://issues.redhat.com/browse/DBZ-7376

@akanimesh7 Applied, thanks


feat: error if ECS task definition includes awslogs driver running in blocking mode

This PR adds a new test (ECSTaskDefinitionAwslogsDriverNotBlocking) which validates whether there are containers in an ECS task definition which are using the awslogs driver in blocking mode. This is added to the AWS Solutions pack as AwsSolutions-ECS8.
Blocking mode is the default configuration, but it is not recommended [^1] [^2] for many workloads as it blocks the container if logs can't be sent to CloudWatch for any reason. This is an availability risk as many applications would rather loose logs during a CloudWatch outage etc. than take downtime.
[^1]: Trusted Advisor check: https://docs.aws.amazon.com/awssupport/latest/user/fault-tolerance-checks.html#amazon-ec2-awslogs-driver-blockingmode
[^2]: Blog post: https://aws.amazon.com/blogs/containers/choosing-container-logging-options-to-avoid-backpressure/

This seems like a good rule, however this is not part of the current AwsSolutions guidance.  Let's move this to the Additional Rules for now.
I'll reach out to the team that manages AwsSolutions rules and see if they want this included in their set of checks.


Don't save back on same files when rotating and cropping leaflet and permit undoing

This would mean that the new image would automatically reload in the browser because they'd have a different name.
The purpose of the undo is to give the staff-editors confidence to fix up leaflets without fear of making mistakes and destroying something permanently.
The functions that do the editing are rotate_image() crop_image() in apps/leaflets/views.py, which are listed as "hacky".
Reason for calling them hacky is that they resize and save the different versions (thumbnail, etc) of the image back into the relevant directories, and repeats the code from within apps/leaflets/models.py UploadSession.handle_file_uploads(), when this ought to be a function from within the LeafletImage model.
The random image filenames of the form "418457ea-55b0-4784-baa7-72df1ed95985.jpg" are coming from the function "update_filename():  return os.path.join(path, str(uuid.uuid4()) + ext)".  [If we used "hashlib.md5(img.tostring()).hexdigest()" we might be able to capture duplicate uploads.]
To make the edits to these images undoable we'll need to add some extra fields into the LeafletImage model, such as image_key_previous, edited_date, edited_staffuser.  Then the leaflet page will trivially be able to show the "last edited", "who by" and "previous picture".
The undo feature will be crude and won't allow redo.
Might add other image processing features, such as sharpen, colour balance, etc, depending on the quality of the photos we are consistently getting.

This issue was moved to DemocracyClub/electionleaflets#71


Use pytest instead of unittest

Use Pytest instead of unittest python module

add pytest==6.2.5 to dev requirements
add pytest settings to setup.cfg - just specify main python code place
update CONTRIBUTING.md about tests run + some description typo fixes in the section
update github actions for running tests
update tests syntax from uniitests to more-pytest style - use asserts instead of self.assert methods

If there is a decision to move to pytest, please consider moving tests into a separate directory as well. (As for the test directory structure, there're strategies as well)

If there is a decision to move to pytest, please consider moving tests into a separate directory as well. (As for the test directory structure, there're strategies as well)

I think that it is a better way too, it can be the second step in s separate PR - for better diff readability
Let's wait for a maintainer opinion

I'm going to move new tests to pytest in the PR and I can move all tests to one folder. Or maybe moving tests to a separate folder will be better in another PR, @Liraim what do you think?

I'm going to move new tests to pytest in the PR and I can move all tests to one folder. Or maybe moving tests to a separate folder will be better in another PR, @Liraim what do you think?

I will move tests to separate folder in another PR. So nothing is required here.


Gwh apidocs 1588 02 add sdk docs

Updates to get the SDK doc files in place so the file sets are better in sync. Plus all broken links in the Japanese are fixed.

This should have been a merge to Japanese branch not main. Closing 225. 227 makes these mods.


Add the ability to access public folders

I wrote this in c#, but for test purposes we would love to do it in Ruby for our cuke tests:
FolderView view = new FolderView(10);
            view.PropertySet = new PropertySet(BasePropertySet.IdOnly);
            //view.PropertySet.Add(FolderSchema.DisplayName);
            SearchFilter searchFilter = new SearchFilter.ContainsSubstring(FolderSchema.DisplayName, folderName);
            view.Traversal = FolderTraversal.Deep;
            FolderId InboxId = new FolderId(WellKnownFolderName.Inbox, "<mySharedFolderName>@<mycompany>.com");
            FindFoldersResults findFolderResults = service.FindFolders(InboxId, searchFilter, view);
            return findFolderResults.Folders[0].Id.ToString();

This let us read emails from a team shared folder.
I dont see a way to do this with the gem currently.
Thank you.
This

How do I access the contacts in a public shared folder? I tried using 'contact.ews_item' but the hash returned doesn't include any of the contact details.


Use sql-client in Flink-1.13 get exception

It looks like flink-1.13 was modified the catalog spi service  to org.apache.flink.table.factories.Factory

https://github.com/tidb-incubator/TiBigData/pull/91/files


Adjust reading of joint grid container to be able to parse hierarchic grid structures.

Depends on: https://github.com/ie3-institute/PowerSystemDataModel/pull/769
The Gridprovider.gridFromConfig method currently does not consider the isHierarchic parameters but just assumes a flat directory hierarchy. This should be changed, after the corresponding option is available in the PSDM

Solved with PSDM 4.0


[External] Codeunit 419 - CreateFileNameWithExtension

Following the advice of @AlexanderYakunin,

please always open new issue if you cannot find something what should be implemented

#1297 has been closed for a while now, but we cannot find the solution in the latest docker images. Can you please verify?

[ ] Solved in BC14

@JesperSchulz

I was going through old, open issues and stumbled over this one. I'm afraid I totally missed this one.
I just checked: the function is exposed in version 15 and newer, but is still local in version 14 and below. As we're not running any SaaS solution on versions lower than 15, I would believe this issue is by now redundant (as you could just expose the function in an on premises installation yourself)? If you'd still like to get the function exposed in version 14, please let me know! Normally we do not backport to that version, but since I left you hang to dry, I would make an exception for you in this case. Would you still like CreateFileNameWithExtension to be made public and external?
My apologies!

Thanks for your suggestion, but the function is indeed not to be backported to earlier versions.