Code generation for Spark ML models into both Java/C++ and CPU/GPU

an optimized C++ deployment environment (using Quora's QMF algo) - possibly including GPUs for batch scoring of multiple requests to take advantage of GPU vectorization

Check out Quora's C-based MF impl:

https://github.com/quora/qmf
https://engineering.quora.com/Open-sourcing-QMF-for-matrix-factorization

an optimized Decision Tree java implementation using Janino code generation against a Spark ML DecisionTree model generated

Optimized Python-based/C++ based environment serving a basic tensorflow algo - perhaps on GPU also.

this should also include codegen for GPUs, or optimized C++ code
this is all possible because of Spark ML 2.0+'s parity between Python and Java/Scala. all models are stored as parquet - and can therefore be saved by any language and loaded by any other language that can read parquet and supports the parquet data types

http://tullo.ch/articles/decision-tree-evaluation/

prototype:
https://github.com/fluxcapacitor/pipeline/tree/master/myapps/codegen/spark/2.0.0/src/main/scala/com/advancedspark/codegen

https://github.com/fluxcapacitor/pipeline/blob/master/myapps/codegen/spark/1.6.1/src/main/scala/com/advancedspark/codegen/spark/ml/classification/DecisionTreeClassificationModelCodeGenerator.scala

Similar to this project https://github.com/jpmml/jpmml-converter/

use gpustat or nvidiasmi command to monitor GPU utilization.

complete


Upload doesn't seem to work in Microsoft Edge

In the console:
[object Error]: {description: "Object doesn't support property or method 'set'", message: "Object doesn't support property or method 'set'", ngDebugContext: Object, ngErrorLogger: function() { [native code] }, number: -2146827850...}
Microsoft Edge 42.17134.1.0

Facing the same issue in IE 11. Can someone help?

Figured this one out. Error: Object doesn't support property or method 'set' is coming because IE 11 doesnot support formData.set() so changed it to formData.append() in all the files of angular-material-fileupload and it started working.


Skip acquiring  GIL for __PYX_XCLEAR_MEMVIEW

This PR Generate following code:

Comparing to master generated code:

Note: Code example is taken from bug report.
Closes #5670

Thanks


Creating my own dataset

Hi @aptx1231 ,
Thanks for this famous research. I am currently researching on the same field only. I have few questions regarding your research.

I want to understand that how you had created the dataset ? Specially the path column in the porto's raw data. H

How you created the porto_eval_enhancedbytrim.csv and porto_eval_enhancedbyshift.csv ? How is it important for the code ?

How did you exported porto_roadmap_edge.geo and porto_roadmap_edge.rel ? Can you please elaborate on it ?

you can refer this link :https://bigscity-libcity-docs.readthedocs.io/zh-cn/latest/user_guide/data/atomic_files.html


correct the systemd file used for ood-portal

the systemd file specified was the ood one not the ood-portal one where these options reside
┆Issue is synchronized with this Asana task by Unito

Hi, thank you for the contribution. I went to check on this and apparently we create both.
[root@68d97ebf4c72 ~]# ls /etc/systemd/system/httpd.service.d/ -lrta
total 16
-r--r--r-- 1 root root   63 Feb 13 16:34 ood.conf
drwxr-xr-x 1 root root 4096 Feb 13 18:03 ..
-r--r--r-- 1 root root  242 Feb 13 18:03 ood-portal.conf
drwxr-xr-x 1 root root 4096 Feb 13 18:03 .

I'm going to have to check in the source as to why this is the case.

Thanks for the contribution! You're right that these docs should point to ood-portal.conf.


2023 Sandbox Project Review - ORAS

2023 Review of ORAS
https://github.com/cncf/toc/pull/943

TOC has decided to change the way annual project reviews for sandbox projects are conducted, and so this review is no longer needed. Closing this issue. Thanks @thschue !


Hotfix 2.0.12 Release Branch Merge to Master (#947)

Running formatter

Extra null check for error, sub_error before calling equals()

Use the yoda conditions we will

Rev version number for MSAL hotfix

Update changelog for 2.0.12 deployment

Rev version to 2.0.12

Supersedes #948

@iambmelt were the basesonctroller changes merged into dev prior to the common version changes?

@iambmelt were the basecontroller changes merged into dev prior to the common version changes?

They were; they were cherry-pick'd into the release/2.0.12 branch, so they are absent from the diff here


Toggle git changes

Would it be possible to toggle file changes on and off? Perhaps in the source control view or individual changes in the diff view.
This would be useful for finding which change caused an issue for example.

An alternative solution is to use the Git: Revert Selected Ranges command (you can set a keybinding for this) to remove a change and cmd+z to enable it again

@sbland, I would suggest that you use the "Revert Selected Ranged" feature as suggested above.


Problem with boolean + GET request method

Hello,
When i send to the API a boolean with methods: POST, PUT; PATCH the value is a boolean.
But when i send to the API a boolean with method GET, the value is a string. I need to test like:
if ($request->myBoolean == 'false') {
}

Is there a solution to this problem?

Finally, i created a small middleware to fix this problem:
<?php

namespace App\Http\Middleware;

use Closure;

class CleanRequestMiddleware {

    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request  $request
     * @param  \Closure  $next
     * @return mixed
     */
    public function handle($request, Closure $next)
    {
        if ($request->isMethod('get')) {

            $params = $request->all();
            $changed = false;

            foreach ($params as $key => $value) {
                if ($value === 'true') {
                    $params[$key] = true;
                    $changed = true;
                } elseif ($value === 'false') {
                    $params[$key] = false;
                    $changed = true;
                }
            }

            if ($changed) {
                $request->merge($params);
            }
        }

        return $next($request);
    }
}


[BEAM-5793] Disable pylint 'self-argument' check

The check currently gets triggered on Jenkins but not when running on MacOS. It
is disabled at other places as well, so I'm assuming this is a valid fix for
now.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

CC @HuangLED @mwylde

Yes, thanks.

Thanks a lot for the fix! :-)

My pleasure :)


Fix symbol crashes.  This pod conflicts with some other libraries

I was getting symbol clashes for Block and BlockContext so I renamed those classes to MGBlock and MGBlockContext.

Heh. I've been waiting for this to happen. And yeah, the rename seems the sensible option.
It'll break backwards compat though, so will need a major version bump.


server lifecycle  timing

https://github.com/nuxt/nuxt.js/issues/11428

@danielroe we have server timings for loading chunks but would be nice to know (particularly where we are performing internal $fetch) what the timings for each handler are

Context and how-to are not really clear to me at the moment but we might benefit from more nitro lifecycle hooks to make such a thing possible.
Related https://github.com/unjs/hookable/issues/37

Is it still not possible to add manual entries for server timing?


net/http: TimeoutHandler does not support Pusher interface

When used as a middleware http.TimeoutHandler breaks HTTP/2.0 Server Push mechanism, as its own writer does not support http.Pusher interface.

@inliquid can you give an example code?

Set up middleware

func setupHandler(r *mux.Router) {
	if cfg.ServerUseGzip {
		server.Handler = alice.New(
			loggerMiddleware,
			throttleMiddleware,
			//timeoutMiddleware,  // <-- breaks Push
			compressMiddleWare,
			recoverMiddleware,
			sessionMiddleware,
			authMiddleware,
		).Then(r)
	} else {
		server.Handler = alice.New(
			loggerMiddleware,
			throttleMiddleware,
			//timeoutMiddleware,   // <-- breaks Push
			recoverMiddleware,
			sessionMiddleware,
			authMiddleware,
		).Then(r)
	}
}

timeoutMiddleware returns http.TimeoutHandler and is quite simple

func timeoutMiddleware(next http.Handler) http.Handler {
	return http.TimeoutHandler(next, time.Duration(cfg.ServerHandlerTimeout)*time.Second, http.StatusText(http.StatusServiceUnavailable))
}

Push something

	p, ok := w.(http.Pusher)
	if ok {
		if err := p.Push("/assets/css/pure/pure.css", &http.PushOptions{
			//Method: "GET",
			Header: http.Header{
				"Accept-Encoding": []string{"gzip"},
			},
		}); err != nil {
			logger.Error.Println("Error pushing pure.css!")
		}
	}

When timeoutMiddleware uncommented ok == false and p == nil

@inliquid Ah sorry, the example can be more simple by using code from blog. I mis-interpreted the response when trying it.
Can you try above commit if it's fixed for you.

Ok, is there any instruction how to patch and test stdlib with this changes applied?

@inliquid I think there's no way unless you re-build your go binary from source.

Thank you for filing this bug @inliquid and @Gnouc for the preliminary questions.
@Gnouc in regards to your question in https://github.com/golang/go/issues/29193#issuecomment-447717573
here is a minimal repro to demonstrate this problem https://play.golang.org/p/e6Pr0U_pMCp or inlined below
package main

import (
	"log"
	"net/http"
	"time"
)

func main() {
	hf := http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
		rw.Write([]byte("OK"))
	})
	handler := http.TimeoutHandler(hf, 55*time.Second, "Timed out!")
	_, ok := handler.(http.Pusher)
	if !ok {
		log.Fatal("TimeoutHandler does not implement http.Pusher")
	}
	log.Print("TimeoutHandler implements http.Pusher")
}

which will print out
TimeoutHandler does not implement http.Pusher

Thanks @Gnouc for the CL, we shall review it for Go1.13.
@bradfitz might you have a wishlist for interfaces that many of these helpers should implement? If so, let's review collaborate on this.

@odeke-em In case we have the wishlist, shall we implement in this issue or raise another one?

@Gnouc great question! So I think we should spin off another issue after we get the wishlist. That way if any rollback needs to be performed, it'll be piecemeal and this CL will stay in solid :)

@bradfitz can you please take a look?


[SPARK-8998][MLlib] Distribute PrefixSpan computation for large projected databases

Continuation of work by @zhangjiajin
Closes #7412

Merged build triggered.

Merged build started.

Test build #39000 has started for   PR 7783 at commit 4ddf479.

@mengxr I've made the lineage changes as requested, but have concerns about the scalability of these changes. See in-line comments.

Merged build triggered.

Merged build started.

Test build #39004 has started for   PR 7783 at commit a61943d.

Test build #39000 has finished for   PR 7783 at commit 4ddf479.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Jenkins test this please

Merged build triggered.

Merged build started.

Test build #158 has started for   PR 7783 at commit a61943d.

Merged build triggered.

Merged build started.

Test build #39010 has started for   PR 7783 at commit a61943d.

Test build #39004 has finished for   PR 7783 at commit a61943d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test build #39010 has finished for   PR 7783 at commit a61943d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test build #158 has finished for   PR 7783 at commit a61943d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

LGTM. Merged into master. Thanks!


Update jest-playwright-preset

Update jest-playwright-preset to fix Webkit issue

Thanks!!

Tests keep hitting Chromium revision is not downloaded. Run "npm install" for all 3 browsers. Do you have any clue?

@FezVrasta I just tried to run test with travis for all browsers. But I also had problems with it. Only chrome works as expected. I'll check it out.

I also encountered this on my local machine a couple of times

@FezVrasta youp! me too.  Deleting node_modules and rerunning yarn again helped. (


Feature/support mirror configuration

Sometimes the address in the bucket becomes invalid or inaccessible
Or it may be slow to visit in some areas, such as China
I modified the url parsing script and now it will parse the mirrirs property in config.json. which is a map type property
Below is my config.jsonfile
{
  "SCOOP_BRANCH": "master",
  "lastupdate": "2021-07-11T10:33:24.5707181+08:00",
  "SCOOP_REPO": "https://github.com/lukesampson/scoop",
  "aria2-enabled": false,
  "list": "all",
  "proxy": "<IP_ADDRESS>:10809",
  "mirrors": {
    "http://repo.msys2.org/": " https://repo.huaweicloud.com/msys2/"
  }
}

At this time, scoop will download files from repo.huaweicloud.com instead of repo.msys2.org

@r15ch13   Can you help me see if there is a problem?
There seems to be something wrong with the automated test script

Please search https://github.com/chawyehsu/dorado for mirrors in China, and submit PR if any need.

What I want is to change the location of the package download through configuration. Instead of downloading the json file in the bucket from gitee or from github
This PR is to replace the url by reading the configuration file


BUG: Enhance fotter of edit profile page

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
The current footer doesn't match the website theme. I would like to change it to like other pages of website. Please assign the task to me. Thank you.
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

I really like your app and want to work on your project ,please assign this issue to me under gssoc-ext 2024

@atmajaa is there any other issue on which i can work.??

@Mansi-Tanwar  try raising an issue for the "How do you prefer to read" pop up. It currently doesn't have a close btn option

@Shalinis19137 Create an issue first. You'll be assigned if it is valid

ok


Pipes in different schemas, but with same name are tainted by each other.

Provider Version
0.17.1
Terraform Version
0.12.x
Describe the bug
Pipes in different schemas, but with the same name are tainted by each other.
In our use case, we use workspaces to segregate our alpha from prod resources. We put our alpha pipes in RAW.ALPHA__SNOWPIPE_S3 and our prod pipes in RAW.SNOWPIPE_S3.
When a run a terraform plan in my prod workspace, it sees the resources in RAW.ALPHA__SNOWPIPE as tainted (likely because they come back in alphabetical order?).
Expected behavior
Pipes should respect schema when checking state.
Code samples and commands
See explanation above.
Additional context
Add any other context about the problem here.

This appears to have been fixed by #291


Failed to create subgroup

I have created a "development" group in which I want to create a "cross" subgroup.
I use the gem in the following way:
gitlab.create_group ('cross', 'development')
Set primary group "development" as path
Which generates the following error:
validate: server responded with code 400: could not save group {: path => ["already took"]} URI request: http: // gitlab / api / v4 / groups (gitlab :: eror: badRequest )
This is OK? may be missing a method to create subgroups?

It was already solved by placing the parend_id in the options


[NNBD] Bugfix: Currencies with high precision

Fix a formatting issue where currencies with precision>2 were outputting weird numbers.
Same as https://github.com/noojee/money.dart/pull/19, but on non-nullable branch.

And this is the one I've manually merged.


Impossible to import in TypeScript project

The package is difficult to import from within TypeScript project.
import defu from 'defu'

Throws:
TypeError: (0 , defu_1.default) is not a function

import * as defu from 'defu'
import defu = require('defu')

Works, but fails TypeScript check:
TS2349: This expression is not callable.
Type 'typeof import("X:/my-project/node_modules/.pnpm/defu@5.0.1/node_modules/defu/dist/defu")' has no call signatures.

const defu = require('defu')

Works, no types, using require.

I added a workaround to<EMAIL_ADDRESS>Can you please try it to see if it fixes typescript issue?

In v6, you can use import { defu } from defu


Signature reader reader does not work in Russian localization

if the client has a localization language Russian then the reader writes that the error of reading the signature.
how fix it?

Russian in "general" is not supported. But there are some essential keyWords translated that helps to map Russian Signature typese.g.

I can scan in the English client tonight.
then switch the client language and scan again
that it would be easier for you to make a parser

i change
init.js
         1: {
                name: '(combat site|kampfgebiet|site de combat|Боевой район)', //*
                label: 'Combat'
            },
            2: {
                name: '(relic site|reliktgebiet|site de reliques|Археологический район)', //*
                label: 'Relic'
            },
            3: {
                name: '(data site|datengebiet|site de données|Информационный район)',
                label: 'Data'
            },
            4: {
                name: '(gas site|gasgebiet|site de collecte de gaz|Газовый район)',
                label: 'Gas'
            },
            5: {
                name: '(wormhole|wurmloch|trou de ver|Червоточина)',
                label: 'Wormhole'
            },
            6: {
                name: '(ore site|mineraliengebiet|site de minerai|Астероидный район)',
                label: 'Ore'
            },
            7: {
                name: '(ghost|Призрачный)',
                label: 'Ghost'
            }

and system_signature.js
 let validSignatureNames = [                                                 // allowed signature type/names
       'Cosmic Anomaly',
       'Cosmic Signature',
       'Kosmische Anomalie',
       'Kosmische Signatur',
       'Anomalie cosmique',
       'Signature cosmique',
       'Космическая аномалия',                                                 // == "Cosmic Anomaly"
       'Скрытый сигнал'                                                    	// == "Cosmic Signature"
   ];

but it doesn't work

@tezdal It probably not works because the JS files are not used in the client as they are. There is a "frontend build" process that bundles/minifies all the JS files.
But this is no problem I´ll add the labels you posted here and do the rest. :) Hopefully I can add the stuff before the next Pathfinder release gets out this weekend.
Thanks!

but i use npm run gulp production and compile product.
These code fragments are still encountered in
public/v1.3.5/app/

FYI 1.4.3 still not work w/ russian signatures :(

@tezdal Thanks for your translation!
I added your suggestions


Consider updating dependency versions

Hi, thanks for the useful plugin!
I noticed the versions of some of this package's dependencies are very old. For example, the read-pkg-up version is from 4 years ago: https://github.com/benmosher/eslint-plugin-import/blob/master/package.json#L114
This can lead to insecure transitive dependencies being brought in for users of this plugin.
For example: eslint-plugin-import > read-pkg-up > read-pkg > normalize-package-data > hosted-git-info@2, which recently had a security alert.

We support eslint 2, and all the node versions it supports, so I’m pretty sure everything is as updated as it could possibly be.

We support eslint 2, and all the node versions it supports, so I’m pretty sure everything is as updated as it could possibly be.

Ah I didn't realize compat for old eslint versions went so far back. That helps explain it, thanks! Would it make sense to to a major version split and drop support for very old eslint versions at some point? Of course you've probably already thought about that, I'm just curious.

There's not much of a benefit to doing that. Most CVEs are false positives anyways, especially in the eslint ecosystem.
For the one you referenced, see #2047.

<EMAIL_ADDRESS>  version "2.0.0"
  resolved "https://registry.yarnpkg.com/pkg-dir/-/pkg-dir-2.0.0.tgz#f6d5d1109e19d63edf428e0bd57e12777615334b"
  integrity sha1-9tXREJ4Z1j7fQo4L1X4Sd3YVM0s=
  dependencies:
    find-up "^2.1.0"

<EMAIL_ADDRESS>  version "3.0.0"
  resolved "https://registry.yarnpkg.com/pkg-dir/-/pkg-dir-3.0.0.tgz#2749020f239ed990881b1f71210d51eb6523bea3"
  integrity sha512-/E57AYkoeQ25qkxMj5PBOVgF8Kiu/h7cYS30Z5+R7WaiCCBfLq58ZI/dSeaEKb9WVJV5n/03QwrN3IeWIFllvw==
  dependencies:
    find-up "^3.0.0"

<EMAIL_ADDRESS><EMAIL_ADDRESS>  version "4.2.0"
  resolved "https://registry.yarnpkg.com/pkg-dir/-/pkg-dir-4.2.0.tgz#f099133df7ede422e81d1d8448270eeb3e4261f3"
  integrity sha512-HRDzbaKjC+AOWVXxAU/x54COGeIv9eb+6CkDSQoNTt4XyWoIJvuPsXizxu/Fr23EiekbtZwmh1IcIG/l/a10GQ==
  dependencies:
    find-up "^4.0.0"

<EMAIL_ADDRESS>  version "5.0.0"
  resolved "https://registry.yarnpkg.com/pkg-dir/-/pkg-dir-5.0.0.tgz#a02d6aebe6ba133a928f74aec20bafdfe6b8e760"
  integrity sha512-NPE8TDbzl/3YQYY7CSS228s3g2ollTFnc+Qi3tqmqJp9Vg2ovUpixcJEo2HJScN2Ez+kEaal6y70c0ehqJBJeA==
  dependencies:
    find-up "^5.0.0"

There's not much of a benefit to doing that. Most CVEs are false positives anyways, especially in the eslint ecosystem.

Increases node_modules size by installing multiple version of same dependencies. There is 10milion of download weekly of eslint-plugin-import pull countless old packages.
Big companies have integrated security scanners into CI/CD systems. It is not fun to explain to your superiors why your project is "yellow"
It actually slows down ESLint process. There is time needed to require all these additional packages.

https://npm.anvaka.com/#/view/2d/eslint-plugin-import 74 nodes
https://npm.anvaka.com/#/view/2d/eslint 113 nodes

node_modules size is irrelevant; disk space is infinite and free.
Yes, i understand that the fact that 99% of CVEs are false positives makes security scanners inconvenient. The solution isn’t to make one-off changes in specific deps, it’s to fix the security industry, or fix your metrics for what’s acceptable.
I would be very shocked if cold start require time mattered to eslint performance significantly. Either way, updating deps would have zero impact on this cost.

node_modules size is irrelevant; disk space is infinite and free.

It is not true. https://paulmillr.com/posts/chokidar-3-save-32tb-of-traffic/
Eslint alone:
$ du -sh ./node_modules/
18M     ./node_modules/

Eslint and eslint-plugin-import
du -sh ./node_modules/
28M     ./node_modules/

10MB is a non-trivial amount.

I would be very shocked if cold start require time mattered to eslint performance significantly. Either way, updating deps would have zero impact on this cost.

eslint is used as IDE/Editor plugin, pre commit hook, cli in CI/CD, webpack loader etc.
On Project with 3 source files, without any imports
Without eslint-plugin-import
yarn run v1.22.10
$ eslint src
Done in 0.40s.

real    0m0,608s
user    0m0,538s
sys     0m0,069s

With eslint-plugin-import
$ time yarn eslint
yarn run v1.22.10
$ eslint src
Done in 0.50s.

real    0m0,702s
user    0m0,639s
sys     0m0,103s

Just loading this plugin would add 100ms to startup time of eslint. Let's estimate that this plugin is loaded 1milion times daily.
This ((0.1 * 1000000 * 365) / 3600) / 24 would account 422 days globally being lost due this plugin.

Yes, i understand that the fact that 99% of CVEs are false positives makes security scanners inconvenient. The solution isn’t to make one-off changes in specific deps, it’s to fix the security industry, or fix your metrics for what’s acceptable.

You do not understand. In big companies, scanners are mandated and often every exception need to be approved by a human.
Scanners are net positive because they force you to update dependencies and properly maintain your application.
We are not an asking to micro optimize this plugin for require times. Just try to follow best practices and update dependencies on regular basis via automated dependabot/renovate.

I do understand; i work in big companies. The only solution is to fix such naive mandating, and persuade your security teams to recognize that the cost of false positives is much higher than the cost of false negatives.
We do follow best practices - all our deps are as updated as they can be.
If you’d like to ensure that those deps follow best practices and backport security fixes to older major lines, then we’d be able to update to those.

@ljharb I have to question why support for ESLint v2 is so important. ESLint v3 was released in 2016 which dropped support for node < 4. The current v7 release of ESLint supports node =v10 and >=v12. Node versions <=v11 are end-of-life, so even the current release of ESLint supports an end-of-life version of node, which should be plenty of historical support. You're essentially supporting a version of node that was marked end-of-life over four and a half years ago.

EOL status and date is irrelevant.

@ljharb that's it? No future updates just to keep the support for EOL versions of ESLint? As pointed above, projects like this need to be kept up to date for a lot of reasons. Your decision is making me rethink the usage of this package on our project.

@mccraveiro Nobody said "no future updates".
The problem is that these specific dependencies decided to be hostile to their consumers, and unnecessarily drop support of older node versions. The future update will be removing these dependencies and replacing them with friendlier ones.
Most CVEs are false positives; literally every single one of them this issue refers to 100% does not apply to this plugin's usage of the packages. If you insist on blindly treating every CVE as a commandment not to use the package, then you'll rapidly find yourself running out of ecosystem packages to use.

The future update will be removing these dependencies

That's a fine solution for this issue. However you closed this issue without a resolution or an explanation about this roadmap. That's much more reassuring and friendly. Thanks for the update!

@mccraveiro Fair enough :-) and np, happy to elaborate whenever needed.
This particular category of problem is exceedingly noisy and one of the primary causes of maintainer burnout, so my temper's a bit short with it. See https://overreacted.io/npm-audit-broken-by-design/


Bucket.find behavior erratic

find's suffix/prefix searching does not always filter results properly.  Debug.  Also, does not paginate results.

Changed search function to sequential 'if in' operators for the time being.  For larger buckets (>500,000 items), both bucket.dir() and bucket.find() should be revisited with table-based tooling.


Task: #1

Issue: #12
Installed Kali Linux in Virtual Box , link is in text file

Hey @ASHTAD123
This is not a valid Issue Number. Kindly recheck it!

Hey @ASHTAD123
Thanks for opening this PR 🚀. Mentor will review your pull request soon and till then, keep contributing and stay calm.
Thanks for contributing in OpenCode'24 ✨✨!

Hey @ASHTAD123
Your PR has been merged 🥳🥳 and you have earned 20 points.
Thanks for contributing in OpenCode'24 ✨✨


Fix for https://relationalai.atlassian.net/browse/RAI-9265 - Failed to create user as it already exists

Fix for https://relationalai.atlassian.net/browse/RAI-9265
Since we were using a common/fixed email address and all the CI/CD workflows use the same account, user-service failed to create the user with fixed email address again because another workflow had already created it in parallel.

The change looks good but now I'm worried we'll somehow end up with a bunch of users if the tests fail to delete them for some reason. Any thoughts on how we can prevent that? Or at least be properly alerted?

The change looks good but now I'm worried we'll somehow end up with a bunch of users if the tests fail to delete them for some reason. Any thoughts on how we can prevent that? Or at least be properly alerted?

User deletion is happening in the tear down. If it fails to delete, then we can return an error exit code from the tear down function so that the test marked as failed. This is one way of tracking it.

The change looks good but now I'm worried we'll somehow end up with a bunch of users if the tests fail to delete them for some reason. Any thoughts on how we can prevent that? Or at least be properly alerted?

User deletion is happening in the tear down. If it fails to delete, then we can return an error exit code from the tear down function so that the test marked as failed. This is one way of tracking it.

Also DeleteUser is happening in the TestUser itself. If that fails the test will fail and so do the workflow.

@larf311 I have added the functionality in the teardown function to return an exit code but then I had to change the way delete database and delete engine was happening. If the tests had already deleted the database or engine the existing teardown function would return a wrong exit code. So, I have added to code to find and delete as it is doing for user and oauth client.


⚠ ClamAV has degraded performance

In 8a71aa7, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 293 ms

Resolved: ClamAV performance has improved in 28a5aca.


Add free-list support to ArrayStore

@geirst @toregge please review very carefully 🙂

Add support for enabling freelists via ArrayStoreConfig. Currently defaults to false.
Small array allocations from freelist reuse both entry ref and array buffer
Large array allocations from freelist reuse entry ref only

@geirst @toregge PTAL. Implementation now simplified in terms of FreeListAllocator and with explicitly added tracking of extra space used


Daily Changes to TMI Cup Campaign Leaderboard

a Paginator of multiple pages
Page 1:
Changes in Scores
Page 2:
Top 10 of Campaign
Page 3-8:
Top 10 of Each of the Maps
Times also present with the player as well as any position changes with previous day.
Will probably take up 6-8 API Requests
Planned as a task at 12am IST
Should also be toggleable and able to change the campaign id through command
Top 10 of the campaign should be available as command

postponed indefinitely. Will decide later if this is required.

Miss' webhooks works perfectly for this, there is no longer a need.


Allow Azure services to access server

Hi,
in the article, you explain "Allow Azure services to access server" applies for all resources inside the Azure boundary, that may or may not be part of your subscription. From the other side in the embedded video they explain that this is gives access to any resource within your own subscription.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f3102127-3223-8b70-a029-bbc33fea53d6
Version Independent ID: 0bd59bd9-753b-39c9-c773-e8c2a0e086e7
Content: Network Access Controls - Azure SQL Database and SQL Data Warehouse
Content Source: articles/sql-database/sql-database-networkaccess-overview.md
Service: sql-database
Sub-service: security
GitHub Login: @rohitnayakmsft
Microsoft Alias: rohitna

What is the question?

There is no question. The video and the article take confusion regarding how "Allow Azure services to access server" do. Regarding the video you grant access to the resources in your subscription(s), regarding the article the resources "may or may not be part of your subscription". Agree that there is a huge difference between statements.

When you set to that level (Allow Azure services to access server), anything in Azure can access it. Like from my subscription, I can access something you operate in yours. The only thing it would disallow is general traffic from www (non-Azure).

@bbojilov Please convey if you have any further question or we can go ahead close this thread,

@bbojilov Its been a while we heard from you.
We will proceed to close this issue now.

Yes, you can close it.


fix generation of man directories

When inspecting the output of a locally-built opam package, I noticed that the shell expansion used to create the man directories wasn't working as expected, so I updated the builder to fix the issue.
Before:
$ tree ./result/share/
./result/share/
└── man
    └── man{1,2,3,4,5,7,8,9}

3 directories, 0 files

After:
$ tree ./result/share/
./result/share/
└── man
    ├── man1
    ├── man2
    ├── man3
    ├── man4
    ├── man5
    ├── man6
    ├── man7
    ├── man8
    └── man9

11 directories, 0 files

Thanks!


Add --ntp-servers to brkt-cli.

--ntp-servers allow customers to configure locally hosted ntp-servers
for time synchronization. This commit makes this option visible.

Looks good, thanks.


make the contact us page responsive

Is your proposal related to a problem?
the contact us page is not responsive for small screens
(Write your answer here.)
Describe the solution you'd like
<!-
i want to make the contact us page responsive-->

please assign this issue to me @subhadipbhowmik

Hi, I’d love to help resolve this issue—could you please assign it to me?
@subhadipbhowmik

Also assign it me me @subhadipbhowmik

Hi, I’d love to help resolve this issue—could you please assign it to me?
@subhadipbhowmik

Please raise your own issue

Hi, I’d love to help resolve this issue—could you please assign it to me?
@subhadipbhowmik

Don't love to resolve it I created a PR and  raise and love your own issue


Skipfirst option in opcpublisher versions above 2.5

Is your feature request related to a problem? Please describe.
I am working on opcpublisher2.8.1 and it seems that "SkipFirst" option has been removed from versions above 2.5.
I am connecting to Ignition OpcServer and it seems that every time I do a restart of publisher, it duplicates the value in my database, which is not expected in subscription mode.
Please suggest if there any other option by which I can achieve the same functionality.

You're right, the SkipFirst option is still missing in 2.8.x. Unfortunately, there is no alternative option to it available.

@anipuramattom  actually the behavior is correct by spec and SkipFirst was a "special" feature added to OPC Publisher < 2.6 to address the side effect you are experiencing.

@hansgschossmann but how we can handle thousands of duplicate messages coming from opc module when ever we restart the module ?

@gibin-francis-maersk there is not really a good work around. The options you have are limited to IoT Edge processing. Routing all OPC Publisher messages to a filter module which tries to detect the connect event might do it. But this can be unreliable since it depends on the quality of the detection logic and adds additional latency, since you need to inspect each event.
Without doubt the right approach is to add back the SkipFirst option.

@hansgschossmann Thanks for the reply, Will Wait for the same if that cane be considered as feature request.

#1924

Implementation is now in main, will be released in 2.9 community preview release next year.


History Sidebar

What's the Problem?
No problem, a request
Solution/Idea
It would be nice to have a History option in the sidebar area. Having the option to turn it on and off would be nice too. Basically it would contain a number (configurable) of recent locations (both local and remote).
Alternatives
n/a
Priorities

Capability
Priority

This proposal will allow developers to accomplish W
Must

This proposal will allow end users to accomplish X
Should

This proposal will allow developers to accomplish Y
Could

This proposal will allow end users to accomplish Z
Won't

Files Version
No response
Windows Version
No response
Comments
No response

Thank you for your feedback, we don't have plans to implement this but we're tracking an issue to add a history page.


Add potentials for rooftop PV

Since we separated low- and high-voltage nodes, we've been optimizing utility PV and rooftop PV separately. The PV installable potentials were set based on land use, but we should develop separate installable potentials for rooftop PV.
According to this JRC paper EU28 rooftops could produce 680 TWh/a but this summary of other JRC work suggests 1500 TWh/a for the EU28?
Are they distinguishing between residential, tertiary and industrial properties?
@martavp Do you have any good sources for this?

Would also impact build_renewable_profiles rule in PyPSA-Eur, right?

Yes you're right. I think the main issue is the distribution of angles to the azimuth etc, which will be varied for rooftop, versus fairly uniform for utility PV.

The usual proxy is to estimate rooftop potential based on population density.
A per-capita rooftop PV potential of 1kW/person can be estimated with reasonable assumptions:
10m2/person * 150W/m2 (15% efficiency) * 0.9 inverter efficiency * 0.75 (of the rooftop space useful) = 1 kW/person
This JRC-TIMES report also assumes 10m2 per capita, and the total rooftop area of 7935 km estimated with the high-resolution geospatial assessment also fits well with that number.
This approach results in a potential capacity of 740 GW, with 1100 equivalent hours, this will produce 800 TWh/a which is similar to the figures that you have found Tom.
For the time series, what ninja does, and what I did in this paper, is assuming that tilt and orientation angles of installations follow two Gaussian distributions whose average are the optimum values (tilt=latitude, orientation=south). This creates some smoothing of the peak capacity factors which slightly increases the smoothing that is already created by spatial integration.

Brilliant, thank you @martavp! @lisazeyen and I have seen big swings between rooftop and utility depending on the grid connection costs, but given that the rooftop potential is lower than half the total PV we see installed with the sector-coupled model, I think we can just max out the rooftop potential and then do the rest utility (also with agrovoltaics). We can ditch the 50-50% split from previous work.

Yes, I agree. Rooftop should be prioritized and Agrivoltaics is the way to go in southern Europe, it saves water and reduces land competition.
In the future, we can estimate the potential of agrivoltaics assuming a lower power density and considering the Corine Land Cover categories 21 (Land principally occupied by agriculture, with significant areas of natural vegetation ) and 22 (Agro-forestry areas ). Those are currently excluded  when the available land for PV is estimated.

The usual proxy is to estimate rooftop potential based on population density.

Corine also includes "Continuous urban fabric" and "Discontinuous urban fabric". Could this somehow be more accurate (or less)?

That will be less accurate since they include areas identified as "urban", but not the rooftop area available.

For calculating the retrofitting potentials per country I was using data from the hotmaps project, which does include also informations about roofs for EU countries (percentage of flat/tilted roofs/ number of dwellings per building type / type of buildings (Multi-family, apartment, office). Maybe one could use also this data to calculate the potential? You can find the data in the branch retrofitting, under data/retro/data_building_stock.csv

Another reference: https://doi.org/10.1016/j.esr.2019.100388, implemented in https://github.com/timtroendle/euro-calliope, uses satellite pictures to assess rooftop potentials. "Two studies assess the technical potential of roof-mounted PV at the continental level, finding potentials of 840 TWh/a [26] and 1500 TWh/a [28]." The potentials in that paper look much higher, but I only find country-level results in the Appendix, not the totals, grrr....

This was done with a simplified assumption of 1 kWp/m² in https://github.com/PyPSA/pypsa-eur-sec/pull/66.


HADOOP-19291. CombinedFileRange.merge should not convert disjoint ranges into overlapped ones

Description of PR
Currently, Hadoop has a bug to convert disjoint ranges into overlapped ones and eventually fails by itself.
INPUT
Range(offset: 13816220, length: 24)
Range(offset: 13816244, length: 7423960)
OUTPUT (from the test case error)
Range(offset: 13816200, length: 100)
Range(offset: 13816200, length: 7424100)
[ERROR] Failures:
[ERROR]   TestVectoredReadUtils.testMergeSortedRanges:267->assertRangeListSize:358 [coalesced ranges]
Expected size:<1> but was:<2> in:
<[range [13816200-13816300], length=100, reference=null; range count=1, data size=24,
  range [13816200-21240300], length=7,424,100, reference=null; range count=1, data size=7,423,960]>

FAILURE
Caused by: java.lang.IllegalArgumentException: Overlapping ranges range [...] and range [...]
	at org.apache.hadoop.util.Preconditions.checkArgument(Preconditions.java:213)
	at org.apache.hadoop.fs.VectoredReadUtils.validateAndSortRanges(VectoredReadUtils.java:316)

How was this patch tested?
Pass the newly added test case.
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
18m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
49m 15s

trunk passed

+1 :green_heart:
compile
19m 22s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
18m 28s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
1m 12s

trunk passed

+1 :green_heart:
mvnsite
1m 38s

trunk passed

+1 :green_heart:
javadoc
1m 17s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 51s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m 48s

trunk passed

+1 :green_heart:
shadedclient
38m 17s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  2s

the patch passed

+1 :green_heart:
compile
18m 59s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
18m 59s

the patch passed

+1 :green_heart:
compile
17m 59s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
17m 59s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m  8s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
mvnsite
1m 39s

the patch passed

+1 :green_heart:
javadoc
1m  8s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 51s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m 47s

the patch passed

+1 :green_heart:
shadedclient
38m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
20m 11s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
1m  2s

The patch does not generate ASF License warnings.

256m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7079

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 273b29dcc033 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 9e6afa03a397736bc3f76e4df583c2208f274be3

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/1/testReport/

Max. process+thread count
1263 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
47m 22s

trunk passed

+1 :green_heart:
compile
19m 19s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
17m 15s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
1m 18s

trunk passed

+1 :green_heart:
mvnsite
1m 46s

trunk passed

+1 :green_heart:
javadoc
1m 17s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 53s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m 39s

trunk passed

+1 :green_heart:
shadedclient
37m 45s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 33s

the patch passed

+1 :green_heart:
compile
18m 33s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
18m 33s

the patch passed

+1 :green_heart:
compile
17m 22s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
17m 22s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
1m 14s
/results-checkstyle-hadoop-common-project_hadoop-common.txt
hadoop-common-project/hadoop-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)

+1 :green_heart:
mvnsite
1m 44s

the patch passed

+1 :green_heart:
javadoc
1m 11s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 51s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
2m 45s

the patch passed

+1 :green_heart:
shadedclient
38m 37s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
20m  8s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

236m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7079

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux d70e2255221f 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / edffb3b68d111befd8d395faa15d0e35de5198f5

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/2/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7079/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

cc @steveloughran

To @saikatroy038 , it depends on the Hadoop's decision. I reported on RC2 vote.

https://lists.apache.org/thread/6dshj3nb26mlhzhmd8xkmw78l3zk896t

HADOOP-19098 (or more) made Hadoop 3.4.1 fails even when users give disjoint ranges like the test example. If we want to keep HADOOP-19098, we need to have this relaxation. Otherwise, I believe we can revert HADOOP-19098 like Hadoop 3.4.0 behavior.
Note that Hadoop 3.4.1 RC2 didn't pass even Apache ORC and Apache Spark unit tests. I guess it will be the same in Apache Parquet unit tests.

To @saikatroy038 , it depends on the Hadoop's decision. I reported on RC2 vote.

https://lists.apache.org/thread/6dshj3nb26mlhzhmd8xkmw78l3zk896t

HADOOP-19098 (or more) made Hadoop 3.4.1 fails even when users give disjoint ranges like the test example. If we want to keep HADOOP-19098, we need to have this relaxation. Otherwise, I believe we can revert HADOOP-19098 like Hadoop 3.4.0 behavior.
Note that Hadoop 3.4.1 RC2 didn't pass even Apache ORC and Apache Spark unit tests. I guess it will be the same in Apache Parquet unit tests.

Thank you for clarifying

This is a tricky issue. So thanks a lot @dongjoon-hyun for finding this.
I have one question though:
Shouldn't validation run on the base ranges? The initial disjoint ranges are getting converted into overlapping ranges because of block size roundup and rounddown.

Now even if there are two separate overlapping ranges later, it is fine as there will be separate read call for both.
We do not want to merge this kind of ranges because populating two overlapping ranges buffer from merged range stream will cause issues.
Which ORC/Parquet test is failing because of this?

I suspect the problem is that as we merge ranges they may overlap and then be rejected by the validation code. So we must not attempt to merge overlapping ranges.

Thank you, All.
From my side, the Hadoop's quantization of ranges is the root cause of this problem and the newly added overlapping rejection code only reveals the issue here. I believe Hadoop 3.4.1 is immature and not tested fully this kind of corner case.
What I expected is very simple. Don't complain when the user gives a set of disjoint ranges like Hadoop 3.4.0.
This PR is only aiming to show the problem. I don't mind dropping this patch if this give you the enough evidence of issues.
I can close this PR if Apache ORC and Spark unit tests passed without any problem.

I can close this PR if Apache ORC and Spark unit tests pass without any problem.
@dongjoon-hyun  Could you please point out which ORC and Spark unit tests case are falling becasuse of this along with logs if possible? Thanks

Okay found it here https://github.com/apache/hadoop/pull/7079

Caused by: java.lang.IllegalArgumentException: Overlapping ranges range [30720-14251520], length=14,220,800, reference=null; range count=1, data size=14,220,225 and range [14251008-29585920], length=15,334,912, reference=null; range count=1, data size=15,334,564
at org.apache.hadoop.util.Preconditions.checkArgument(Preconditions.java:213)
at org.apache.hadoop.fs.VectoredReadUtils.validateAndSortRanges(VectoredReadUtils.java:316)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.readVectored(RawLocalFileSystem.java:323)
at org.apache.hadoop.fs.BufferedFSInputStream.readVectored(BufferedFSInputStream.java:182)
at org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:307)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readVectored(ChecksumFileSystem.java:466)
at org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:307)
at org.apache.orc.impl.RecordReaderUtils.readDiskRangesVectored(RecordReaderUtils.java:590)
at org.apache.orc.impl.RecordReaderUtils$DefaultDataReader.readFileData(RecordReaderUtils.java:114)
at org.apache.orc.impl.reader.StripePlanner.readData(StripePlanner.java:178)
at org.apache.orc.impl.RecordReaderImpl.readStripe(RecordReaderImpl.java:1306)
at org.apache.orc.impl.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:1346)
at org.apache.orc.impl.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1389)
at org.apache.orc.impl.RecordReaderImpl.(RecordReaderImpl.java:367)

First merge is called https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L453
and then readVectored https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L466 which will validate again.

Seeing the history validation of overlapping ranges was not there for LocalFS vectored IO but only for S3.

maybe we should say "ok for overlapping ranges but we don't merge?"

Debug this by running the failing ORC tests in IDE today. As I was suspecting, it's the chunk size of 512 > 1 in case of CheckSumFileSystem is causing the disjoint ranges to become overlapping.
Now that we are doing a validation again as compared to branch-3.4.0 https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java#L101 it fails.
I don't think merging logic is broken but the double validation of initial and merged ranges is the problem. Reverting the above line to previous should be the solution.
Also discussed with @HarshitGupta11 and maybe we can write a new algorithm for merging which could handle double validation.

+1 to cutting the validation in raw local; the way the nio reads are executed it is safe.
I should note that it only surfaces with file:// ; ChecksumFS + RawLocal, with a large enough file that multiple large ranges to be supplied close to/adjac
We missed this because we were only doing small files in the unit tests and testing against HDFS and S3 in the full stack tests. "our tests were too good".

FYI, @mukund-thakur and @steveloughran . The remaining failures of RC3 are the following three. Will take a look later.
[ERROR] Failures:
[ERROR]   ChunkReadUtilTest.testReadAll:60 expected: <true> but was: <false>
[ERROR]   ChunkReadUtilTest.testReadAlternateWMinSeekSize:78 expected: <true> but was: <false>
[ERROR]   ChunkReadUtilTest.testReadAlternateWMinSeekSizeDrop:88 expected: <true> but was: <false>
[INFO]
[ERROR] Tests run: 14, Failures: 3, Errors: 0, Skipped: 0


[main] Upgrade to latest dependencies

Cron -knative-prow-robot
/cc knative-sandbox/eventing-writers
/assign knative-sandbox/eventing-writers
Produced by: knative-sandbox/knobots/actions/update-deps
Details:
bumping knative.dev/eventing 19935b1...c02e63a:
  > c02e63a upgrade to latest dependencies (# 6267)
  > 5fcc285 only reconcile triggers that point to the MTChannelBroker class (# 6265)
  > 08ac6d0 upgrade to latest dependencies (# 6266)
bumping knative.dev/pkg 721abec...3a4cc56:
  > 3a4cc56 Add configmap informer factory (# 2466)
  > 13bdd74 add popany to bucketer (# 2445)
  > 616f1ab Add genreconcile for NetworkPolicy. (# 2465)

/test pull-knative-sandbox-eventing-kafka-broker-unit-tests


Set ChatBloc page size to 12 when the Material Theme is enabled

No content

Issue created by BlueBubbles Bot in the BlueBubbles commnity on behalf of @zach in #support.

Done. my beta branch


NTP status page not honouring NTP ACLs

Redmine issue https://redmine.pfsense.org/issues/7609

I had I thought about adding a break when an IP match, this to stop the loop as soon it found a match improving performances.
It all depends on the philosophy of the first vs last match win. Either won't impose any restriction, but since the ACL page can't reorder any entry, for now, I guess it's better to let the last win.


Feature/taxonomy endpoint

(continues work from https://github.com/CenterForOpenScience/osf.io/pull/6080)
Purpose
Add taxonomies/plos endpoint for getting PLOS subjects
Changes
Add view, initialization, model, and serializer for reading in PLOS subjects. Add .json file for storing flat taxonomy locally, to be read in ensure_taxonomies()
Ticket
https://openscience.atlassian.net/browse/PREP-4

Be sure to run API tests locally before pushing. Pass done. 🐧


Signing API: MultipleObjectsReturned: get() returned more than one Version -- it returned 4!

more info: https://sentry.prod.mozaws.net/operations/olympia-prod/group/153985/
MultipleObjectsReturned: get() returned more than one Version -- it returned 4!

Stacktrace (most recent call last):

  File "django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "django/views/decorators/csrf.py", line 57, in wrapped_view
    return view_func(*args, **kwargs)
  File "rest_framework/views.py", line 399, in dispatch
    response = self.handle_exception(exc)
  File "rest_framework/views.py", line 396, in dispatch
    response = handler(request, *args, **kwargs)
  File "amo/decorators.py", line 167, in wrapper
    return f(*args, **kw)
  File "signing/views.py", line 43, in inner
    return fn(view, request, addon=addon, **kwargs)
  File "signing/views.py", line 116, in get
    version = addon.versions.get(version=version_string)
  File "django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "django/db/models/query.py", line 313, in get
    (self.model._meta.object_name, num))

oops, dupe of https://github.com/mozilla/olympia/issues/953


Improve error handling on trigger fire, do not delete trigger mark it paused instead

Currently the trigger is being deleted from the DB on certain error scenarios
We need to improve how do we handle each use case, I think there are 3 use cases when firing

Trigger has being removed from OpenWhisk, Controller is reachable and returns Not Found (404)
Errors that are not recoverable (wrong auth 403, etc..), then delete trigger from db
Errors that with a retry can be succesfully fire on a retry, but never delete the trigger from db, and instead mark the trigger doc in db "paused: true"
others?

paused state for triggers is something that could be use when redudancy providers is implemented

proposed design is the same for all 3 providers kafka, cloudant, alarms provided here https://github.com/openwhisk/openwhisk-package-kafka/issues/167#issuecomment-296223109


fix: dont open a feed for a replication stream that has been destroyed

Related to https://github.com/joehand/hypercloud/issues/16
This one was a tricky one!
There was a case where a feed's storage was being closed prematurely. This led to writes failing. Replication would then stall (see https://github.com/mafintosh/hypercore/issues/54) but did appear to complete successfully if the connection was re-established.
Why was the FD closing prematurely? It's possible that the protocol handshake will uncover an extraneous connection, which needs to be destroyed. Sometimes one of those destroyed connections would be the first to enter the open() logic in this repo. The cleanup() check would see that the stream is destroyed, and that there are no other active streams, and so would call feed.close(). Afterward, other non-destroyed streams would continue processing, and try to write into the feed causing the failure.
It looks like there's no harm in just ignoring open() for any stream that's already destroyed, so that's what this PR does.

wow nice catch!

Thanks! merging and publishing

Published 3.2.3


updates values.py with 2019 Subaru Impreza ecu

Added 2019 Subaru Impreza ECU to enable vehicle recognition. Tested on fork of open pilot master on 11/4.
Validation done on successful drive with openpilot fork.
Route
Route: 78c45ff47fdd934b|2022-11-04--18-34-57
https://github.com/commaai/openpilot/pull/26385

upstream PR commit is already included in current subaru-community branch


highlightWeekDays prop

feat: added highlightWeekDays prop to highlight all dates under a weekday

@devhus Looking fine, however, can you run npm run lint and create pull again?

@devhus Looking fine, however, can you run npm run lint and create pull again?

done.

@Jasenkoo Do i need to create another PR or recreate my fork?

@devhus No need, I will merge it today.


[stable/concourse] Updated values file comments.

Concourse has changed out the separator that works for approle in vault, the = has been replaced by a :. I also updated the refernce link as the Concourse docs page has changed. This is a response to issue #14341.
Signed-off-by: Jason Morgan<EMAIL_ADDRESS>
What this PR does / why we need it:
The documentation in the values file is out of date and misleading.
Which issue this PR fixes

fixes #14341

Special notes for your reviewer:
No code has been changed.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/ok-to-test

thank youuu!


gRPC SubscribeTransactions ignores its request argument

Background
The current behaviour of SubscribeTransactions is that it allows passing in a GetTransactionsRequest argument, however completely ignores it.
This is an issue if one expects to e.g. be able to replay confirmed transactions by passing in a request with startHeight and endHeight.
My question is if this is left as an open issue for the future, where SubscribeTransactions should be able to handle such requests; or if SubscribeTransactions should never be expected to handle the request parameter and it should therefore be removed from its function signature?
Steps to reproduce
Calling SubscribeTransactions with a GetTransactionsRequest containing e.g. startHeight and endHeight.
Expected behaviour
Epected behaviour would be for SubscribeTransactions to replay confirmed transactions respecting the request parameters, and then also listen to any notifications for new transactions.
Actual behaviour
SubscribeTransactions ignores the request parameter and only listens for new transactions.

My question is if this is left as an open issue for the future, where SubscribeTransactions should be able to handle such requests; or if SubscribeTransactions should never be expected to handle the request parameter and it should therefore be removed from its function signature?

The request parameters were likely added after the fact and were not handled for SubscribeTransactions. I'd say they should be handled instead of removing them, though in the meantime we should return errors when these parameters are set to make it clear to users that they are not being handled.

Thanks for the clarification @wpaulino !

I'll take a look at this.

@ErikEk Me too actually. Let me know if you want to share ideas or something :)

@ErikEk Me too actually. Let me know if you want to share ideas or something :)

Ok. Np. I'll let you do it. Will let you know if I figure something out.

Just chiming in that we ran into the same issue, there's a workaround with making a separate GetTransactions call but it would be nice (and less confusing) if the start_height param were honored (although I guess end_height doesn't really make any sense in the context of SubscribeTransactions).


Microblink ViewController is not presenting

Reason
We can't login
Description
The microblink receive an viewController to present the two steps verification, but when they try to present this view, an error is throwing with this message
Attempt to present <BRNewStaticScanViewController: 0x10f83ac00> on <TtGC7SwiftUI19UIHostingControllerGVS_15ModifiedContentVS_7AnyViewVS_12RootModifier_: 0x10c034e00> (from <TtGC7SwiftUI19UIHostingControllerGVS_15ModifiedContentVS_7AnyViewVS_12RootModifier_: 0x10c034e00>) which is already presenting <TtGC7SwiftUI19UIHostingControllerV11TikiRewards10HomeScreen: 0x10c02f400>.
Success Criteria

[ ] show the two steps verification view

Additional Information
No response

this feature will be removed


Consider creating a microservices key

Current situation
We are experiencing some issues with projects containing different repos inside.
Let's make an example.
PROJECT A
--> microservice 1
--> microservice 2
...
--> microservice n

Right now our solution is to tell PAs to create a new repo, let's call it metarepo, in order to store the information regarding the whole project. This metarepo will contain the publiccode.yml file which will be indexed in the catalog.
However, this solution has some flaws:
a) if the metarepo does not have a clear README file, the information regarding the other repositories (containing the n microservices) will be rather vague or lost. This is suboptimal.
b) the vitality index is calculated exclusively based upon the information regarding the metarepo and this does not make sense.
Proposal
We should add a key, called for example components or microservices, containing the list of repositories "related" to the project. As such, the metarepo will still contain the publiccode.yml file inside but each repo will be somehow listed in the catalog. This could also facilitate the calculation of the vitality index.

Great proposal, I would like to say that the components seperately are
'codebases' or 'products' in their own right, so them having a
publiccode.yml in the file seems right to me.
Perhaps we can tell people that the metarepo, and especially the README
in that can serve as an important entrypoint for the project and give them
a reason to work on that. This readme could include things such as high
level overview, the business cases etcetera.
On Tue, 18 Jun 2019 at 11:03, libremente<EMAIL_ADDRESS>wrote:

Current situation
We are experiencing some issues with projects containing different repos
inside.
Let's make an example.
PROJECT A
--> microservice 1
--> microservice 2
...
--> microservice n
Right now we are encouraging PAs to create a new repo, let's call it
metarepo, in order to store the information regarding the whole project.
This metarepo will contain the publiccode.yml file which will be indexed
in the catalog.
However, this solution has some flaws:
a) if the metarepo does not have a clear README file, the information
regarding the other repositories (containing the n microservices) will be
rather vague or lost. This is suboptimal.
b) the vitality index is calculated exclusively based upon the
information regarding the metarepo and this does not make sense.
Proposal
We should add a key, called for example components or microservices
containing the list of repositories "related" to the project. As such, the
metarepo will still contain the publiccode.yml file inside but each repo
will be somehow listed in the catalog. This could also facilitate the
calculation of the vitality index.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/italia/publiccode.yml/issues/62?email_source=notifications&email_token=AAOUFPNRWKND36QCXNMNXWDP3CQGHA5CNFSM4HY523N2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G2CSOUQ,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOUFPJGAGLK2JW4WBIYNZLP3CQGHANCNFSM4HY523NQ
.

the components seperately are 'codebases' or 'products' in their own righ

@bvhme I understand your point of view and I believe this is true for well built standalone microservices or components which can be deployed "independently" from the other pieces. However, we are facing some products which have several not really independent pieces so I fear that inserting a publiccode.yml in each of those repo would generate confusion when browsing the catalog. I don't have a clear solution to this yet.

This readme could include things such as high level overview, the business cases etcetera.
Exactly, this is our view of a metarepo.

Hi there from the city administration of Münster, Germany, and thanks for publiccode.yml!
We are starting to use the  publiccode.yml for two current projects (and all other future projects, too, of course) and are facing this exact problem.
One project has a frontend and a backend in two repositories. They are not to be used indepently, though.
It's similar with a different project that splits up to three repositories: source code, documentation to rebuild hardware and design files.
What to do with these project structures? One publiccode.yml per project in the "main" repo (whatever this is)? The same publiccode.yml in every repo?
(Referencing the corresponding issues: https://github.com/bCyberGmbH/leezenflow-doku/issues/2 , https://github.com/bCyberGmbH/leezenflow-design/issues/1 , https://github.com/reedu-reengineering-education/smart-city-dashboard-backend/issues/4 )

Hi all, this is also an issue I expect we will be facing with many source code from the French public sector.
I am in favor of the metarepo approach.
I'm tempted to use dependsOn to list the repositories that contribute to the metarepo, but I'm not sure entries in dependsOn can have a URL - if not, that's certainly a problem.  (Also, this suggestion relies on my understanding of dependsOn as something different than the list of libraries one can find in e.g. package.json.  I don't think adding the list of dependencies on publiccode.yml is a good idea.  But this example suggests that my interpretation is okay.)
Another (complementary?) suggestion is to add a key to declare that a publiccode.yml belongs to a metarepo. Something like publiccodeYmlType: "meta" - but that name is not good enough IMHO.
I would refrain from adding a microservices key, as systems may have various architectures, this would lead to create more keys.  If dependsOn cannot be used to declare contributing repos, I'd use something neutral like "components" - or anything that cannot be confused with dependencies/libraries/packages/etc.

@Smart-City-Muenster it's a pleasure to have you here! I'm happy you're finding publiccode.yml useful!
I agree with your concerns and I also see what @bzg is saying. I think that having a components could indeed be a useful key. So far we've decided to either promote a repository as "main" repository, or simply create a new lightweight one which refers to the various components (see as an example https://github.com/italia-software/gitlab).
I agree that a machine-readable solution would be preferrable, and I love the name components for it! Would you like to open a Pull Request adding the key to the spec? We can then work on refining the details of it and get it approved


OneToMany documentation

Good morning. I found, IMHO, an issue in documentation.
I tryed the code provided here: OneTomany
But I get compiler error. In effect the  working code for me is the following:
var users = db.FetchOneToMany<UserDto>(x => x.Cars,y => y.UserId,      "select u.*, c.* from Users u inner join Cars c on u.UserId = c.UserId order by u.UserId");
Thanks

I suppose you are using version 3, while the documentation you are referring to is for version 2.
Version 3 documentation: Version 3. There are some breaking changes.

Thanks for quick response. Yes i'm using version 3 and did not realize that documentation was for version 2.

I will try and get the doco updated. Feel free to submit a PR for it if u
want to help out.


feat: add Calcom template

Add Calcom template
Docker compose from their Github
Different logos available on their website, included in this pr the white text svg.

Some variables did not set up as I thought they would like $SERVICE_BASE64_CALCOM_KEY. I do not grasp all of the magic yet, looking for feedback if anyone is testing this.

I've managed to make the instance start with this compose.yaml:
version: '3.8'
services:
  calcom:
    image: 'calcom/cal.com:latest'
    environment:
      - 'DATABASE_URL=postgres://${SERVICE_USER_POSTGRES}:${SERVICE_PASSWORD_POSTGRES}@postgres:5432/${POSTGRES_DATABASE:-calcom}'
      - 'DATABASE_DIRECT_URL=postgres://${SERVICE_USER_POSTGRES}:${SERVICE_PASSWORD_POSTGRES}@postgres:5432/${POSTGRES_DATABASE:-calcom}'
      - 'WEBAPP_URL=${SERVICE_FQDN_CALCOM}'
      - 'NEXTAUTH_URL=${SERVICE_FQDN_CALCOM_API_AUTH}'
      - 'NEXTAUTH_SECRET=${SERVICE_BASE64_32_SECRET}'
      - 'CALENDSO_ENCRYPTION_KEY=${SERVICE_BASE64_32_SECRET}'
      - 'NEXT_PUBLIC_WEBAPP_URL=${SERVICE_FQDN_CALCOM}'
      - 'NEXT_PUBLIC_API_V2_URL=${SERVICE_API_V2_URL}'
      - 'CALCOM_TELEMETRY_DISABLED=${CALCOM_TELEMETRY_DISABLED}'
      - 'EMAIL_SERVER_HOST=${SMTP_HOST}'
      - 'EMAIL_SERVER_PORT=${SMTP_PORT}'
      - 'EMAIL_SERVER_USER=${SMTP_USER}'
      - 'EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}'
      - 'EMAIL_FROM=${MAIL_FROM}'
    depends_on:
      postgres:
        condition: service_healthy
    command: "sh -c \"\n  npx prisma migrate deploy --schema /calcom/packages/prisma/schema.prisma &&\n  npx prisma db seed --schema /calcom/packages/prisma/schema.prisma &&\n  npm run start\n\"\n"
    healthcheck:
      test:
        - CMD
        - curl
        - '-f'
        - 'http://localhost:3000'
      interval: 30s
      timeout: 10s
      retries: 5
    restart: unless-stopped
  postgres:
    image: 'postgres:16-alpine'
    environment:
      - 'POSTGRES_USER=${SERVICE_USER_POSTGRES}'
      - 'POSTGRES_PASSWORD=${SERVICE_PASSWORD_POSTGRES}'
      - 'POSTGRES_DB=${POSTGRES_DATABASE:-calcom}'
    volumes:
      - 'calcom-postgresql-data:/var/lib/postgresql/data'
    healthcheck:
      test:
        - CMD-SHELL
        - 'pg_isready -U ${SERVICE_USER_POSTGRES} -d ${POSTGRES_DATABASE:-calcom}'
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
volumes:
  calcom-postgresql-data: null

However, this instance does not starts "as new", skipping the first user creation...
I'm then unable to connect to it 😅
Anyone has tried something?

Here is my working version:
version: '3.8'
services:
  calcom:
    image: 'calcom/cal.com:latest'
    environment:
      - 'DATABASE_URL=postgres://${SERVICE_USER_POSTGRES}:${SERVICE_PASSWORD_POSTGRES}@postgres:5432/${POSTGRES_DATABASE:-calcom}'
      - 'DATABASE_DIRECT_URL=postgres://${SERVICE_USER_POSTGRES}:${SERVICE_PASSWORD_POSTGRES}@postgres:5432/${POSTGRES_DATABASE:-calcom}'
      - 'ALLOWED_HOSTNAMES=["${SERVICE_FQDN_CALCOM}"]'
      - 'NEXTAUTH_URL=${SERVICE_FQDN_CALCOM_API_AUTH}'
      - 'NEXTAUTH_SECRET=${SERVICE_BASE64_32_SECRET}'
      - 'CALENDSO_ENCRYPTION_KEY=${SERVICE_BASE64_32_SECRET}'
      - 'NEXT_PUBLIC_WEBAPP_URL=${SERVICE_FQDN_CALCOM}'
      - 'NEXT_PUBLIC_API_V2_URL=${SERVICE_API_V2_URL}'
      - 'CALCOM_TELEMETRY_DISABLED=${CALCOM_TELEMETRY_DISABLED}'
      - 'EMAIL_SERVER_HOST=${SMTP_HOST}'
      - 'EMAIL_SERVER_PORT=${SMTP_PORT}'
      - 'EMAIL_SERVER_USER=${SMTP_USER}'
      - 'EMAIL_SERVER_PASSWORD=${SMTP_PASSWORD}'
      - 'EMAIL_FROM=${MAIL_FROM}'
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test:
        - CMD
        - curl
        - '-f'
        - 'http://localhost:3000'
      interval: 30s
      timeout: 10s
      retries: 5
    restart: unless-stopped
  postgres:
    image: 'postgres:16-alpine'
    environment:
      - 'POSTGRES_USER=${SERVICE_USER_POSTGRES}'
      - 'POSTGRES_PASSWORD=${SERVICE_PASSWORD_POSTGRES}'
      - 'POSTGRES_DB=${POSTGRES_DATABASE:-calcom}'
    volumes:
      - 'calcom-postgresql-data:/var/lib/postgresql/data'
    healthcheck:
      test:
        - CMD-SHELL
        - 'pg_isready -U ${SERVICE_USER_POSTGRES} -d ${POSTGRES_DATABASE:-calcom}'
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
volumes:
  calcom-postgresql-data: null

EDIT: Added the ALLOWED_HOSTNAMES=["${SERVICE_FQDN_CALCOM"] to allow apps to be added (tested with Calendar ICS, which is working as intended).
Removed WEBAPP_URL as deprecated and replaced by NEXT_PUBLIC_WEBAPP_URL wich is already present.
I'm still having some issues (something related to the auth, as I'm regularly getting redirected to the localhost:3000 instead of the next_auth_url).

I have tried to run calcom on coolify as well. Nor the compose.yml nor the .env-exemple are good to deploy via docker-compose on coolify. Actualy, your template got me the farthest. I see 2 green services, I click to access the link and for a fraction of a second, I can spot the logo and auth-initial login screen, but then the system turns black.

There is another PR open https://github.com/coollabsio/coolify/pull/3503 maybe you guys can work it out together but both of your templates did not work properly in my testing so it would be great if you could fix it and let me know when it works, thanks so much.

Thank you for the different feedback, here is an updated version. Completed the onboarding, connected an ICS calendar and tested login/logout without additional configuration.
Please let me know in more details if you have any issues

Thank you for the PR 💜. I will test it and fix it if necessary.

Hi @FranckKe  I might be late on my comment but could you consider an update to enable external database usage? As of my last test, this service depends on a PostgreSQL database that is created internally. What if I already have a PostgreSQL database I want to use? How can I configure it?
Fyi @peaklabs-dev

@DesmondLouis You can just remove the Postgres DB form the compose file and add the details of your external DB to the env variabels. Make sure you connect cal.com to the same network as your DB to make it work.


Exe file does not launch correctly

The maze runner.exe file, when launched does not launch and outputs the error message "cannot find mazerunner.info"

Now fixed


City Bloxx is not supported

I like this game very much, hope you can support it.

Can you provide the jar file,thank you.

tower_bloxx_240x320-77975.zip

Sorry , I can not support the jar because it use the m3g module which it not support by this project now, but i can provide another jar game which is similar to this , I will upload it to the website later.
[都市摩天楼.zip](https://github.com/user-attachments/files/17089197/default.zip)

Sorry , I can not support the jar because it use the m3g module which it not support by this project now, but i can provide another jar game which is similar to this , I will upload it to the website later. [都市摩天楼.zip](https://github.com/user-attachments/files/17089197/default.zip)

Thank you , where did you find this game that works.

Sorry , I can not support the jar because it use the m3g module which it not support by this project now, but i can provide another jar game which is similar to this , I will upload it to the website later. [都市摩天楼.zip](https://github.com/user-attachments/files/17089197/default.zip)

Thank you , where did you find this game that works.

in this website https://iniche.cn/

Sorry , I can not support the jar because it use the m3g module which it not support by this project now, but i can provide another jar game which is similar to this , I will upload it to the website later. [都市摩天楼.zip](https://github.com/user-attachments/files/17089197/default.zip)

Thank you , where did you find this game that works.

in this website https://iniche.cn/

Great, will you support games that require m3g module in the future?

Sorry , I can not support the jar because it use the m3g module which it not support by this project now, but i can provide another jar game which is similar to this , I will upload it to the website later. [都市摩天楼.zip](https://github.com/user-attachments/files/17089197/default.zip)

Thank you , where did you find this game that works.

in this website https://iniche.cn/

Great, will you support games that require m3g module in the future?

It is difficult because m3g use some native modules, if I can convert those modules to javascript ,  I will support it , it has a long way to go.


feat: Integrate osmosis-api into price-feeder

Description
closes: https://github.com/umee-network/osmosis-api/issues/42

Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...

[ ] included the correct type prefix in the PR title
[ ] added ! to the type prefix if API or client breaking change
[ ] added appropriate labels to the PR
[ ] targeted the correct branch (see PR Targeting)
[ ] provided a link to the relevant issue or specification
[ ] added a changelog entry to CHANGELOG.md
[ ] included comments for documenting Go code
[ ] updated the relevant documentation or specification
[ ] reviewed "Files changed" and left comments if necessary
[ ] confirmed all CI checks have passed

Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...

[ ] confirmed the correct type prefix in the PR title
[ ] confirmed all author checklist items have been addressed
[ ] reviewed state machine logic
[ ] reviewed API design and naming
[ ] reviewed documentation is accurate
[ ] reviewed tests and test coverage
[ ] manually tested (if applicable)

🔥 🔥 🔥 🔥 🔥

Figured out that the increasing OSMO price over time was caused by the provider candles map pointer being passed and written to by the convert to usd logic in the price-feeder (since the quote for OSMO is ATOM this resulted in the OSMO candle price being multiplied by the price of ATOM every tick). I've made this fix for the MEXC and Crypto provider as well.


Ensured all Activities stored in transcript have Ids

Fixes #3665

As, done in JS's TranscriptMiddlewareLogger already, ensure that all cloned activities in logger have Ids, so that when  logged into MemoryTranscriptStore, each Item also has an Id.
If an activity doesn't have an id, MemoryTranscriptStore will generate an Id for that activity/Item for the transcript storage, prefixing it with "g_" for "generated", as done in JS.
Context

Not all Activities created have Ids assigned to them (Activity spec in botframework-sdk)
ContinuationToken within the MemoryTranscriptStore.GetTranscriptActivitiesAsync relies on last Activity.Id of loaded PagedResult
Because of the above 2 bullets, if an Activity without an Id is the last item in the PagedResult, we may end up infinitely loading the same PagedResult over and over again.

Testing
In order to test that activities with null ids are assigned ids properly in transcript storage, I had to create a custom TestAdapter, since the one "off the shelf" automatically fills in an activity id, never allowing for null ids to pass through to the TranscriptMiddlewareLogger layer.

Would have created an own separate class, however needed to derive from it, since TestFlow is tightly coupled to TestAdapter (it requires an adapter of specifically type TestAdapter instead of an interface)

Well then...
@mrivera-ms thank you for reviewing.
All of your points were valid!
Just got merged before I pushed the changes though 😲


Major performance regression in latest master build of pilot

Updated some instances of pilot to master. The bad ones are on 57f05da9544b3ed56de0ef039d2f39296f38134a, good ones on 6ca06fc7b6858f7454adbc74bd64d67a5e22a12a.
Commits to pilot in this window:
* 3360dba13 "Shriram Rajagopalan"<EMAIL_ADDRESS>10 hours ago
| MeshNetworks: support gateways with node ports (#22885)
* bf473222c "Zhonghu Xu"<EMAIL_ADDRESS>25 hours ago
| Fix debug inject (#22923)
* 3ab4d7c7c "Zhonghu Xu"<EMAIL_ADDRESS>35 hours ago
| crd client cleanup (#22921)
* 1d3b13632 "John Howard"<EMAIL_ADDRESS>2 days ago
  Fix revision CR scoping (#22876)

Strongly suspect #22885

did a bad job cropping this but that is 33% of the CPU, normally its ~0%

Reverting #22885 fixes this

Fixed


In HA mode, if you do not truncate consumerOffsetMap, it may cause a bug

BUG REPORT

1.nodes in syncStatusSet  all not available.
2.enableElectUnCleanMaster = true, slave3 was chosen to be master.
3.slave4 tried to truncate commitLog.
4.slave4  was chosen to be master and continued to append new message to commitLog.
5.consumer started to consume message from consumerOffset.

Good catch!  I think we'd better correct consumerOffset while truncating the commitlog. Could you submit a pull request to fix the issue?

I believe this is a design flaw. Leader/master nodes shall ONLY replicate their WAL logs to their peers/slaves.  All activities, including updates of consume-offsets, should be reflected in the WAL log. Peer nodes and slaves restore state machines by replaying WAL.
The current implementation has more than one replication path, this is the root cause of the discrepancy.

We should avoid using EnableElectUncleanMaster.


Change in metadataSecretName does not redeploy scheduler

Checks

[X] I have checked for existing issues.
[X] This report is about the User-Community Airflow Helm Chart.

Chart Version
1.11.0
Kubernetes Version
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.9", GitCommit:"d15213f69952c79b317e635abff6ff4ec81475f8", GitTreeState:"archive", BuildDate:"2023-12-20T00:00:00Z", GoVersion:"go1.21.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"28", GitVersion:"v1.28.6-gke.1289000", GitCommit:"a53df8308e54c8cea2e8cf44be1842106e93d04f", GitTreeState:"clean", BuildDate:"2024-01-26T09:25:53Z", GoVersion:"go1.20.13 X:boringcrypto", Compiler:"gc", Platform:"linux/amd64"}

Helm Version
3.11.1

Description
I had a working Airflow deployment, connected to a CloudSQL Instance.
I then created a new CloudSQL Instance, restored a Backup into this new instance and changed the Kubernetes Secret with the Connection string to point to this new Instance.
The webserver correctly picked up the new secret, but the scheduler was not restarted with the new config. In the Webserver under Admin -> Configs I could see the new DB Connection being present.
Only after manually deleting the scheduler pod and re-applying the helm chart the scheduler was re-created and started with the correct DB connection.
Relevant Logs
No response
Custom Helm Values
airflow:
  executor: KubernetesExecutor
  ingress:
    web:
      enabled: True
      ingressClassName: nginx
      hosts:
        - name: airflow.OUR.TLD
          tls:
            enabled: true
            secretName: tls-cert
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt
  postgresql:
    enabled: false
  data:
    metadataSecretName: connection-string
  networkPolicies:
    enabled: True
  config:
    scheduler:
      min_file_process_interval: 10
    webserver:
      expose_config: True
    providers.jdbc:
      allow_driver_class_in_extra: True
      allow_driver_path_in_extra: True
  extraEnv: |
    - name: AIRFLOW__CORE__LOAD_EXAMPLES
      value: 'True'

Update: the triggerer Pod is also not redeployed

This issue is related to the official chart, not the community chart, please raise it on the airflow/airflow repo.


Need to dissoc "open_id" key from tokeninfo from audit logs

here: https://github.com/zalando-stups/friboo/blob/fbfedf39dea957c6f2b51c58d4bb2d4a9ad26b55/src/org/zalando/stups/friboo/system/audit_log.clj#L69
"open_id" key contains JWT tokens and should not be logged

Still needs to be released in a new version on Maven central.


expose column kwarg in append!

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/36939
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.7 -m "<description of version>" 3fddec755085bdd4bc383e471470507f3b492860
git push origin v0.1.7


Beiwe GPS data

How is the GPS data recorded?
The GPS data has large gaps in it which makes me think that perhaps only large changes are recorded which helps save on storage space.
This could be why in my initial analysis using the GPS data, I was finding wildly different distances from the participant's home to their coordinates. Perhaps I should be looking at only coordinates where significant time was spent. Then from there, I can compare those locations to their home's location.
Tasks

[x] Look at raw GPS data from the Beiwe server for multiple participants to confirm these gaps
[x] Contact Cameron/Anibal about how the GPS data are recorded

Extension
We need to get an estimate as to what constitutes a large enough change in distance to register in the GPS data.

[x] Determine distances between subsequent Lat/Long points

Looked at GPS data and noticed the gaps although they aren't as large as I initially suspected. Still, I am emailed Cameron and Anibal to see how often GPS data are recorded.

Cameron replied and said that there IS some threshold and if you don't move far enough, then the GPS coordinates do not change. The example in the documentation says 500 meters, however this value needs to be tested.

Looks like there are plenty of distances between points that are less than 50 meters. The next step would be to see if the timestamps are consistent between smaller distances (meaning that the GPS just logs at a certain rate unless there is a big gap in distance).

From my analysis, I cannot seem to find any distance that causes Beiwe to record the GPS coordiantes.


Allow custom item in the menu

[x] Add support for custom element (not entity) in the menu
[ ] Mark custom menu as selected when clicked
[ ] Add tests
[ ] Add documentation

Refs #343

I'll use another approach.


this._map._renderer is undefined

I'm using the latest version of Leaflet and the Canvas renderer. I've copied the code from the docs.
Please help! :)

I can confirm.
Uncaught TypeError: this._map._renderer is undefined
    setText http://localhost/.../public/js/leaflet.textpath.js:78
    _textRedraw http://localhost/.../public/js/leaflet.textpath.js:43
    _updatePath http://localhost/.../public/js/leaflet.textpath.js:36
    _update http://localhost/.../public/js/leaflet.js:5
    _reset http://localhost/.../public/js/leaflet.js:5
    onAdd http://localhost/.../public/js/leaflet.js:5
    onAdd http://localhost/.../public/js/leaflet.textpath.js:18
    _layerAdd http://localhost/.../public/js/leaflet.js:5
    whenReady http://localhost/.../public/js/leaflet.js:5
    addLayer http://localhost/.../public/js/leaflet.js:5

It seems that it only happens if you're using panes. Without panes, it works.


fix: shrinkSecret to actually shrink

Currently shrinkSecret is only spending CPU cycles and returning the input object unchanged. Fix to return the shrunk object
May help a little with memory usage

Wow good catch! Thanks a lot for your contribution @dpiddock :smiley:
Cheers


Add instructions for how to use adversaries with Sabaki

I also created a public google drive folder to host our adversaries at https://drive.google.com/drive/folders/1-bGX-NQOh6MuRPoXJgYHb9-jWRJvviSg?usp=sharing.

Tom's already reviewed this so I'm not going to review it by default, but feel free to tag me if I should look at all/some of it.

@ed1d1a8d are you planning on making any changes to this, or is it just blocked on my/Tom's review?

Still planning on making some changes here. Will re-request review when ready.

Thanks for the review Tom. I made your requested changes and also updated the main README to be a bit more clean and to link to the sabaki instructions. Re-requesting review.


feat(probe): implement mount probe

Implement mount probe to fetch mount related information like filesystem, mount point etc.

Codecov Report

Merging #248 into master will increase coverage by 0.04%.
The diff coverage is 56.31%.

@@            Coverage Diff            @@
##           master    #248      +/-   ##
=========================================
+ Coverage   52.05%   52.1%   +0.04%
=========================================
  Files          49      51       +2
  Lines        2261    2332      +71
=========================================
+ Hits         1177    1215      +38
- Misses        996    1029      +33
  Partials       88      88

Impacted Files
Coverage Δ

cmd/ndm_daemonset/probe/probe.go
100% <ø> (ø)
:arrow_up:

...m_daemonset/controller/disk_to_device_convertor.go
0% <0%> (ø)
:arrow_up:

cmd/ndm_daemonset/probe/mountprobe.go
0% <0%> (ø)

pkg/mount/mountinfo.go
0% <0%> (ø)

cmd/ndm_daemonset/probe/udevprobe.go
65.85% <100%> (+2.52%)
:arrow_up:

cmd/ndm_daemonset/controller/blockdevice.go
80.3% <100%> (+0.3%)
:arrow_up:

pkg/udev/mockdata.go
71.42% <100%> (+0.37%)
:arrow_up:

pkg/udev/common.go
91.93% <100%> (+12.57%)
:arrow_up:

cmd/ndm_daemonset/filter/osdiskexcludefilter.go
64.86% <100%> (ø)
:arrow_up:

cmd/ndm_daemonset/controller/disk.go
89.01% <90.9%> (+0.63%)
:arrow_up:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b919f9c...72afc42. Read the comment docs.


hooks files are "skipped" if a folder with the same name exists

Describe the bug
If you have a hooks.server.ts or hooks.ts and you create a folder with the same name (hooks.server or more commonly hooks) the respective file will be "ignored" during the server generation.
This is because here
https://github.com/sveltejs/kit/blob/78404dfe1eb346723eefc183278b85f25485b419/packages/kit/src/core/sync/write_server.js#L85-L119
we call resolve_entry with config.kit.files.hooks.server and config.kit.files.hooks.universal which by default are src/hooks.server and src/hooks. However inside resolve_entry we check if it's a directory and if it's a directory we resolve index on that directory
https://github.com/sveltejs/kit/blob/78404dfe1eb346723eefc183278b85f25485b419/packages/kit/src/utils/filesystem.js#L169-L191
What we should do is probably check also for the file and if there's a file AND a directory give precedence to the file.
Reproduction

go here
check that only the hooks.server.ts handle function is logging to the terminal
delete the hooks folder
check that now also the reroute function is logging to the console
optionally create a folder named hooks.server (you need to refresh the page for it to take fully effect...bug in sveltelab sorry 😅)
check that now no hook is logged

Logs
No response
System Info
System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 18.20.3 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.15.6 - /usr/local/bin/pnpm
  npmPackages:
    @sveltejs/adapter-auto: ^3.0.0 => 3.3.1
    @sveltejs/kit: ^2.5.27 => 2.9.0
    @sveltejs/vite-plugin-svelte: ^4.0.0 => 4.0.2
    svelte: ^5.0.0 => 5.4.0
    vite: ^5.4.4 => 5.4.11

Severity
serious, but I can work around it
Additional Information
This can be very hard to debug.

Wondering if the solution to this would solve https://github.com/sveltejs/kit/issues/12065 too


Questions about img_size (224, 224) and (336, 336)

Tks for your great work!
I noticed the resize_pos function in internlm-xcomposer-7b
https://huggingface.co/internlm/internlm-xcomposer2-7b/blob/main/build_mlp.py#L68-L105
it seems that the code builds on the basis: "There are only 224 * 224 images that are supported, which means if the img_size is 224 in finetune.sh, the resize_pos in the init func of CLIPVisionTower will be executed, which also means that if the img_size in finetune.sh is not set to be 224, the finetune can not be executed successfully(I noticed this PR)"
I want to check if my judgements above are right?
If so, I am curious about the purpose of resize operation, because the vit model is set to be openai/clip-vit-large-patch14-336 in the code, which indicate the pos_embedding actually is for (336, 336) scaled images. but the resize_pos function just resize the pos_embedding layer to support the (224, 224) scale images instead of supporting the origin (336, 336) with openai/clip-vit-large-patch14-336  modeling? Or if you want to support (224, 224), why not use something like openai/clip-vit-large-patch14-**224** as a pretrained model?

We select different image resolutions for different models. The img_size is 224 for the -7b model, 490 for the -vl-7b model and not necessary for the -4khd-7b model.
We select openai/clip-vit-large-patch14-336 since it is a stronger visual encoder than other variants.

Tks for your reply! So since the embedding layer is trainable when sft, the img_size actually doesn't affect the performance though the pretrained support for 336 * 336 in clip-vit-large-patch14-336 is abandoned?

During our pre-training stage, both the vision encoder and Partial LoRA are fine-tuned, which will mitigate your concern about img_size misalignment.

During our pre-training stage, both the vision encoder and Partial LoRA are fine-tuned, which will mitigate your concern about img_size misalignment.

OK, Thanks for your reply!