Need GeoSpark best practices in AWS Environment

Hi,
Looking for best practices or examples to run GeoSpark in AWS Environment. I have tried GeoSpark SQL and GeoSpark Core in EMR and seeing scalability and performance issues.
I have created a below Scala program and executed in EMR. In my desktop (i7-7700 CPU, 32 GB RAM) for 1 million Points and 19k polygons, it is taking 18 minutes, where as in EMR (1 master node and 1 core node, which is double capacity than desktop) is taking 30-50 min. I tried different options in EMR but still not able achieve the best performance. Can you please let me know is there any best practices to run the GeoSpark is Cluster?
import org.apache.log4j.{Level, Logger}
import org.apache.spark.serializer.KryoSerializer
import org.apache.spark.storage.StorageLevel
import org.apache.spark.{SparkConf, SparkContext}
import org.datasyslab.geospark.enums.{FileDataSplitter, GridType, IndexType}
import org.datasyslab.geospark.formatMapper.shapefileParser.ShapefileReader
import org.datasyslab.geospark.spatialOperator.{JoinQuery, KNNQuery, RangeQuery}
import org.datasyslab.geospark.spatialRDD.{CircleRDD, PointRDD, PolygonRDD}
import java.text.SimpleDateFormat
import java.util.{Calendar, Date}
import org.apache.spark.sql.SparkSession
import org.datasyslab.geospark.serde.GeoSparkKryoRegistrator
import org.datasyslab.geosparksql.utils.Adapter
import org.datasyslab.geosparksql.utils.GeoSparkSQLRegistrator

  Logger.getLogger("org").setLevel(Level.WARN)
  Logger.getLogger("akka").setLevel(Level.WARN)

  var sparkSession:SparkSession = (SparkSession
    .builder()
    .config("spark.serializer",classOf[KryoSerializer].getName)
    .config("spark.kryo.registrator", classOf[GeoSparkKryoRegistrator].getName)
    .config("spark.driver.memory", "1g")
    .config("spark.executor.memory", "2g")
    .config("spark.sql.shuffle.partitions", "200")
    .config("spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version",true)
    .config("spark.shuffle.service.enabled", "true")
    .enableHiveSupport()
    .appName("AuctionGeoSparkTestProcess")
    .getOrCreate())

GeoSparkSQLRegistrator.registerAll(spark)

  val resourceFolder = "s3://vish-bigdata-prod-temp/data/"

  val PointRDDInputLocation = resourceFolder + "points.tsv"
  val PointRDDSplitter = FileDataSplitter.WKT
  val PointRDDIndexType = IndexType.RTREE
  val PointRDDNumPartitions = 16
  val PointRDDOffset = 0

  val PolygonRDDInputLocation = resourceFolder + "polygons.tsv"
  val PolygonRDDSplitter = FileDataSplitter.WKT
  val PolygonRDDNumPartitions = 16
  val PolygonRDDStartOffset = 0
  val PolygonRDDEndOffset = -1

  val joinQueryPartitioningType = GridType.QUADTREE
  val eachQueryLoopTimes=5

val startTime = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(java.util.Calendar.getInstance().getTime())
println(startTime )

    val polygonRDD = new PolygonRDD(sparkSession.sparkContext, PolygonRDDInputLocation, PolygonRDDStartOffset, PolygonRDDEndOffset, PolygonRDDSplitter, true,PolygonRDDNumPartitions)
    val pointRDD = new PointRDD(sparkSession.sparkContext, PointRDDInputLocation, PointRDDOffset, PointRDDSplitter, true, PointRDDNumPartitions, StorageLevel.MEMORY_ONLY)

    pointRDD.spatialPartitioning(joinQueryPartitioningType)
    polygonRDD.spatialPartitioning(pointRDD.getPartitioner)

    pointRDD.buildIndex(PointRDDIndexType, true)

    pointRDD.indexedRDD.persist(StorageLevel.MEMORY_ONLY)
    polygonRDD.spatialPartitionedRDD.persist(StorageLevel.MEMORY_ONLY)

    for(i <- 1 to eachQueryLoopTimes)
    {
      val resultRDDCount = JoinQuery.SpatialJoinQuery(pointRDD, polygonRDD, true, false).count()
      println(resultRDDCount)
    }

  val endTime = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(java.util.Calendar.getInstance().getTime())
  println(endTime)

Thanks
Vish

@viswanatharv

Try to use GridType.KDBTREE and IndexType.QUADTREE
Before

pointRDD.spatialPartitioning(joinQueryPartitioningType)
polygonRDD.spatialPartitioning(pointRDD.getPartitioner)

Add
pointRDD.repartition(XXX)

XXX should double your current pointRDD partition number
For example
pointRDD.repartition(1000)
pointRDD.spatialPartitioning(joinQueryPartitioningType)
polygonRDD.spatialPartitioning(pointRDD.getPartitioner)

@jiayuasu why quadtree over R-TREE? When would you recommend an R-Tree?

@geoHeil According to Figure 15 a and b in GeoSpark paper:http://www.public.asu.edu/~jiayu2/geospark/publication/GeoSpark_Geoinformatica_2018.pdf
Quad-Tree index outperforms R-Tree index. However, only R-Tree supports KNN query. So, if you do KNN query, you have to R-Tree

Thanks for the update. I am not seeing repartition() method in pointRDD. I have checked in 1.1.3 and SNAPSHOT and not seeing this method. Can you please let me know where can I find this method.
Thanks
Vish

@viswanatharv Sorry, my bad. That should be
pointRDD.rawSpatialRDD.repartition(1000)

@jiayuasu, after all the changes, using desktop and aws cluster is able to complete the task in 18 minutes. Thanks for the great info.
And can you please share if you have any other performance improvement techniques.

@viswanatharv, you mentioned AWS environment.  Are you using EMR?  Can you pass along any information on how you bootstrapped GeoSpark to your cluster?  We are looking for any help to get us going on a POC.  Thanks<EMAIL_ADDRESS>

@bpurdy1645, we did the POC on EMR but in production we are using Glue service. Above mentioned code with few changes mentioned by @jiayuasu is added to the production code. Still seeing performance issues and sometimes rather than taking 10 min, the jobs is taking 4 hrs and not finishing. Not sure what other techniques needs to use to improve the performance...Can somebody please provide the details about how to improve the performance in cluster?

@jiayuasu, Hope you are doing well. We have changed above code (last few lines) as mentioned below.
val resultRDDCount = JoinQuery.SpatialJoinQueryFlat(pointRDD, polygonRDD, true, false)
resultRDDCount.saveAsTextFile(resourceFolder + "market_data")
We are using above code to check points exists in polygons and writing data to s3 with some additional columns and also writing polygons column data to the s3. The size of the s3 for the folder is really huge with polygons. Compare to without polygons the folder is small (like 60MB vs 600GB). Problem here is writing huge data to s3 and also subsequent processes are taking little extra time to process huge data. One way to avoid polygons column is to convert it to data frame. Tried to use Adapater.toDf() and it is expecting [Geometry,Geometry] but not [Polygon,Point]. Is there any method available to convert RDD to data frame or skip the polygon data to write to s3 ?
Thanks
Vish

@viswanatharv The reason why you got [Polygon, Point] type join result is that you use Typed SpatialRDD which is not recommended now. You should use Spark DataFrame / ShapefileReader / WKB reader, WKT reader / GeoJSONreader to create a generic SpatialRDD.
The document has provided several clear examples. https://datasystemslab.github.io/GeoSpark/tutorial/rdd/#from-sparksql-dataframe
Here is a full coding example:
https://github.com/DataSystemsLab/GeoSpark/blob/master/sql/src/test/scala/org/datasyslab/geosparksql/adapterTestScala.scala#L135

@bpurdy1645 Did you succeed to bootstrap GeoSpark on EMR?
If so, could you please share instructions.

@bpurdy1645, Below are the bootstrap details to load GeoSpark and other configuration. This needs to be go to EMR launch site under Step 1: Software and Steps / Edit software settings / Enter Configuration. Please let me know if you need more details.
[
{
"classification":"spark-defaults",
"properties":{
"spark.executor.memory":"1g",
"spark.driver.memory":"8g",
"spark.shuffle.compress":"true",
"spark.executor.heartbeatInterval":"120s",
"spark.network.timeout":"240s",
"spark.executor.memoryOverhead":"1g",
"spark.shuffle.service.enabled":"true",
"spark.kryo.registrator":"org.datasyslab.geospark.serde.GeoSparkKryoRegistrator",
"spark.serializer":"org.apache.spark.serializer.KryoSerializer",
"spark.jars":"s3://vish-bigdata-prod-temp/jars/GeoSpark/geospark-1.2.0.jar,s3://vish-bigdata-prod-temp/jars/GeoSpark/geospark-sql_2.3-1.2.0.jar"
}
}
]

@viswanatharv Were you able to get this code running in AWS Glue at some point?

Hi @zachwarren4, the jobs were configured in AWS Glue and its running in production. Please let me know if you need any details?

I am working on it now, did you need to do anything special except for include the jars?

@zachwarren4, Not required any special settings. Regular Glue job settings are required. And I didn't used jar files for the actual code, I have used Scala code directly in Glue. And the object name should be GlueApp. Please let me know if you need more details.

Thanks for the help! Much appreciated.


Error: Config problem

Describe the bug
After installing organize-tool with no errors and running the command in the README
pip install -U organize-tool
organize new

the following error is encountered
Error: Config problem
Cannot find config "/home/user/.config/organize/config.yaml"

Was the folder and config file supposed to have been created automatically but something went wrong?
Environment (please complete the following information):

OS: Opensuse tumbleweed
Output of organize --version: organize v3.2.1

Your config file
# paste your config file here

Getting same error on Windows-- tried manually creating config.yaml, still does not work

Same error message on MacOS

Thanks everyone for the reports! Will be fixed in the next version.

Update is released 👍


id1 is throwing an error when there's no result

Try: ^id1 base 6871
Traceback (most recent call last):
  File "/home/bot/tsubaki/prod/lib/python3.8/site-packages/discord/ext/commands/bot.py", line 903, in invoke
    await ctx.command.invoke(ctx)
  File "/home/bot/tsubaki/prod/lib/python3.8/site-packages/discord/ext/commands/core.py", line 859, in invoke
    await injected(*ctx.args, **ctx.kwargs)
  File "/home/bot/tsubaki/prod/lib/python3.8/site-packages/discord/ext/commands/core.py", line 94, in wrapped
    raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'NoneType' object has no attribute 'is_low_priority'

So turns out id1 flips out when it can't distinguish between 2 cards, such as the 2 forms of Miada. We're no longer developing for id1 so this is a wontfix but we are wrapping in a try-catch with an error message explaining to use id3 for the query.


Respect limitRanges when creating pods

What type of PR is this?
/kind bug
What this PR does / why we need it:
Controller must respect range limits when creating pods and setting resource requirements otherwise such pods won't be admitted.
Which issue(s) this PR fixes:
Fixes https://github.com/tnozicka/openshift-acme/issues/113
Does this PR introduce a user-facing change?:

Controller now respects RangeLimits when creating exposer pods.

@rbo when this gets green, would you mind verifying this fixes your issue?

Yes, of course!

LGTM
Build images via quay.io:

quay.io/rbo/openshift-acme-controller:respect-range-limits
quay.io/rbo/openshift-acme-exposer:respect-range-limits

My "test log"
oc expose svc/web \
  --hostname="examples-stage.openshift.pub" \
  --name="examples-stage-openshift-pub"

# create cname elb.6923.rh-us-east-1.openshiftapps.com.

oc apply -f https://raw.githubusercontent.com/rbo/openshift-acme/respect-range-limits/deploy/single-namespace/serviceaccount.yaml
oc apply -f https://raw.githubusercontent.com/rbo/openshift-acme/respect-range-limits/deploy/single-namespace/role.yaml
oc apply -f https://raw.githubusercontent.com/rbo/openshift-acme/respect-range-limits/deploy/single-namespace/issuer-letsencrypt-staging.yaml
oc create rolebinding openshift-acme --role=openshift-acme --serviceaccount="$( oc project -q ):openshift-acme" --dry-run -o yaml | oc apply -f -
oc apply -f - <<EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: openshift-acme
  labels:
    app: openshift-acme
spec:
  selector:
    matchLabels:
      app: openshift-acme
  replicas: 2
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: openshift-acme
    spec:
      serviceAccountName: openshift-acme
      containers:
      - name: openshift-acme
        image: quay.io/rbo/openshift-acme-controller:respect-range-limits
        imagePullPolicy: Always
        args:
        - --exposer-image=quay.io/rbo/openshift-acme-exposer:respect-range-limits
        - --loglevel=4
        - --namespace=\$(CURRENT_NAMESPACE)
        env:
        - name: CURRENT_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
EOF

oc annotate route/examples-stage-openshift-pub kubernetes.io/tls-acme=true

curl -k -vvv -I https://examples-stage.openshift.pub/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to examples-stage.openshift.pub (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=examples-stage.openshift.pub
*  start date: Apr 14 13:32:49 2020 GMT
*  expire date: Jul 13 13:32:49 2020 GMT
*  issuer: CN=Fake LE Intermediate X1
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> HEAD / HTTP/1.1
> Host: examples-stage.openshift.pub
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx/1.16.1
Server: nginx/1.16.1
< Date: Tue, 14 Apr 2020 14:45:36 GMT
Date: Tue, 14 Apr 2020 14:45:36 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 50640
Content-Length: 50640
< Last-Modified: Tue, 14 Apr 2020 11:41:43 GMT
Last-Modified: Tue, 14 Apr 2020 11:41:43 GMT
< ETag: "5e95a177-c5d0"
ETag: "5e95a177-c5d0"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Set-Cookie: 174addd57126df3343b83661dde6cc34=76612db3d40b919faab4a95b1572381a; path=/; HttpOnly; Secure
Set-Cookie: 174addd57126df3343b83661dde6cc34=76612db3d40b919faab4a95b1572381a; path=/; HttpOnly; Secure
< Cache-control: private
Cache-control: private

<
* Connection #0 to host examples-stage.openshift.pub left intact
* Closing connection 0

thanks @rbo
/lgtm

/hold
CI got borked by being switched to another cluster (https://github.com/tnozicka/openshift-acme/pull/118)

/hold cancel
the CI fix merged https://github.com/tnozicka/openshift-acme/pull/118

/retest

/retest

/retest

/retest


Pidee board

The pidee board looks interesting. I have a couple (sent one to you, @waveform80). It features eight dip switches, a push button and three LEDs. Would be good to provide a class for it.

Component
pin

button
18

dip 1
6

dip 2
5

dip 3
25

dip 4
24

dip 5
23

dip 6
22

dip 7
27

dip 8
17

red led
13

green led
12

yellow led
19

Thoughts:

Dip switches are a perfect example of ButtonBoard
Dip switches are numbered 1-8 on the board, so we might want to offset the index in the API somehow
Having the push button as [0] and the dip switches as [1:8] would be a cunning trick but that would break things like all() and any
LEDs are traffic light colours but not in traffic light order (red, green, yellow) so might not make sense to use TrafficLights object for these, just a generic composite

Hi Ross from The Workers here. This is great
We are changing the LEDs from RGY to RGB in the next batch. RGB makes it easier to distinguish the LEDs as all on when looking a them from a distance.

The (not yet merged) ButtonBoard is based on CompositeDevice, which means as well as doing:
dipswitches = ButtonBoard(6,5,25,24,23,22,27,17)
print(dipswitches[0].pressed)
print(dipswitches[7].pressed)

you can also do:
dipswitches = ButtonBoard(dip1=6, dip2=5, dip3=25, dip4=24, dip5=23, dip6=22, dip7=27, dip8=17)
print(dipswitches.dip1.pressed)
print(dipswitches[7].pressed) # this style still works

(note that I've missed out _order here, because alphabetical sorting is the order we want anyway)
I've typed this in from memory, so apologies if I've got some of the syntax wrong.
So the implementation of a Pidee class might be something like:
class Pidee(CompositeDevice):
    def __init__(pwm=False):
        LEDClass = PWMLED if pwm else LED
        super(CompositeDevice, self).__init__(
            button=Button(18),
            dipswitches=ButtonBoard(dip1=6, dip2=5, dip3=25, dip4=24, dip5=23, dip6=22, dip7=27, dip8=17),
            red=LEDClass(13),
            green=LEDClass(12),
            yellow=LEDClass(19),
            _order=('button', 'dipswitches', 'red', 'green', 'yellow')
        )

Or given that it contains LEDs, perhaps it should inherit from CompositeOutputDevice rather than CompositeDevice?

Ok so now ButtonBoard is in, we can probably throw something together based on @lurch's ideas above.
Note that ButtonBoard.pressed is not implemented. Instead, I think tuple(ButtonBoard.value) or similar must be used.

There is is_pressed, if that helps? https://github.com/RPi-Distro/python-gpiozero/blob/master/gpiozero/boards.py#L194

Hi @rc1 did the new revision of the Pidee board get made?


KAFKA-5054 : synchronized all the methods

This avoids inconsistent results while querying or storing the underlying storage

retest

Thanks for the PR @lijubjohn -- I was reading the ticket, but I am not sure if I fully understand the issue to be honest. The ticket mentions IQ and that putIfAbsent and delete do two operation -- however, IQ only allows read access anyway, and thus I don't understand.
@dguy You created the original ticket. Can you help out here?
Also, for IQ we actually synchronized store access state store level (ie, RocksDBStore and InMemoryKeyValueStore already.

@mjsax @lijubjohn I think here is the issue @dguy reported:
Consider the putIfAbsent call:
        final byte[] previous = get(key);  // 1
        if (previous == null) {                    // 2
            put(key, value);
        }
        return previous;

It is possible that a concurrent get call is triggered between 1 and 2. The IQ query will return none while at the end it the key will have some values instead; similar situations for delete.
But thinking about this, this could be just treated as "the IQ get call was triggered before the putIfAbsent", since we do not guarantee strict time-ordering for such concurrent operations anyways. So I think we do not necessarily need to fix the JIRA.

@mjsax if you agree we can close the JIRA as not a bug.

I agree with this. The result must be the same as an serial execution (not specified with one), and this seems to be guaranteed as you pointed out.

Hi @lijubjohn as discussed about I'm going to close the ticket as not a bug. Please feel free to pick up other tickets from https://issues.apache.org/jira/issues/?jql=project %3D KAFKA AND resolution %3D Unresolved AND component %3D streams ORDER BY priority DESC
thanks for your contributions again!


Model trivially infeasible but returns Optimal status

What version of OR-Tools and what language are you using?
Version: 9.3.10497
Language: C#
Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
CP-SAT
What operating system (Linux, Windows, ...) and version?
Ubuntu 21.10
What did you do?
Steps to reproduce the behavior:

Load the model
Set StringParameters of the CpSolver to "stop_after_first_solution:true"
Run Solve
It outputs the following:
Model trivially infeasible, variable 76 has lower bound 29 and LowerOrEqual() was called with an upper bound of 28 Model trivially infeasible, variable 76 has lower bound 31 and LowerOrEqual() was called with an upper bound of 30 Model trivially infeasible, variable 76 has lower bound 30 and LowerOrEqual() was called with an upper bound of 29

What did you expect to see
If the model is truly infeasible, the return value should be CpSolverStatus.Infeasible.
What did you see instead?
The return value of Solve is CpSolverStatus.Optimal
Also, does variable 76 refer to the actual index in the model proto? It's strange because that one has a domain [0, 64].

So does the call to Solve really return the optimal solution? That sounds odd given the stop_after_first_solution parameter.

best_bound == objective_value. It seems optimal.
Laurent Perron | Operations Research | @.*** | (33) 1 42 68 53
00
Le ven. 29 avr. 2022 à 22:51, Pavel Kotrč @.***> a
écrit :

So does the call to Solve really return the optimal solution? That sounds
odd given the stop_after_first_solution parameter.
—
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/3272#issuecomment-1113715995,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3JQU7H4PIQDDQWMWXLVHRDTVANCNFSM5UW4PQBQ
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

OK, thanks for the clarification.


getInitialProps to component props

Is there any way to passing value of getInitialProps to suitable component props like Next.js or After.js?
Thank you!

You mean like passing each component/page its getInitialProps?
I considered this in #6. One drawback of only having an App.js instead of a /pages directory that we can scan is that we don't have access to pages independently unless we try to recreate your component tree or wrap React Router's components.
But honestly, this is only a problem if your using Rogue with REST and need to fetch data per page. But with Grapqhl, each componen queries data it needs so receiving props per page is not necassary; so using getInitialProps is primarily for configuring SSR providers, handling page redirects, etc.

Ah gotcha, that's the intended behavior and drawback. It's kinda sad not having that feature but it's okay as long as we have another solution like using redux or apollo.
Can I ask more questions?
Why it seems that examples with razzle and rogue cli have different behavior?
With rouge cli :

With razzle :

SSR_DATA in rouge cli is updated when route component has getInitialProps, but not in razzle.
Is this intended?
Thank you

Btw, if necessary you can always pass props from the App.js component to each respective page; you can namespace the props to make that easier too

that's the possible solution I think.
hahaha, okay, will push the repo to reproduce the issue, by the way I will open PR to fix some typo.
Thank you @alidcastano!


fix(router): properly handle async router configurations

This change moves handling of async router config callbacks into Router.configure to resolve a race condition when using multiple viewPorts. Async configuration was previously handled externally in app-router and route-loading, but that code failed to block subsequent calls from configuring multiple times. Router.configure() now handles async callbacks, setting isConfigured immeditely to allow callers to check state.
BREAKING CHANGES:

Router.configure() now always returns a Promise, and is no longer chainable.
Router.configure() now applies the configuration in a Promise continuation, so it is not safe to assume that the router will be configured synchronously when the configuration callback is synchronous.

fixes #213

@bryanrsmith Sounds good. Feel free to merge it in and continue on :) Thanks!

Any update on when this will be included in the release? There are a few projects on hold (talked on the Aurelia Gitter) for the next release with it's bug fixes.


Group 2 -- Completed Project 1

Group 2
Christopher Cordero
Bheeni Garg
Brian Wu

Your solution makes good use of clear variable names so the code is very readable. I also like how you modified LANGUAGES to be a dictionary with the name of each language as the key which makes it simpler to access than having LANGUAGES be a list of dictionaries.
I only have two suggestions which you can find as comments in the Files changed tab of the PR and otherwise, great job!


Add working examples and tests for all supported connection types

Description
What is the current behavior?
Currently, we don't have any working example(s) of bigquery connections, because we thought the extra is broken. But apparently we did not configure it right.
Prior to Airflow 2.3, the extra has to be of type string and also in case of bigquery all of the extras needed to be prefixed with extra__google_cloud_platform__. Another example is snowflake where the extra needed to be a json as well to work for airflow 2.2.5 which we support.
closes: #1277
What is the new behavior?
We add examples and tests for all currently supported connection types. Moreover configure the ci pipeline to be able to test them.
Does this introduce a breaking change?
No.
Checklist

[ ] Created tests which fail without the change (if possible)
[ ] Extended the README / documentation, if necessary

@feluelle would there be a way for us avoiding having issues with connection extras changes in future? Does Airflow have some strategy to verify if the users' extras are adequate for the hooks we're using ATM?

@tatiana what do you mean exactly? We are using test_connection to verify that the connections works, in case it doesn't the plan of this issue is to also print the error message to the user. Sometimes it is useful, sometimes not :/


support react 15.5.0

Using the prop-types package to remove depreciation warning from React package.

I want this


Proposal: Decentralized Identity (DID) and Verifiable Credentials

[Decentralized Identity (DID) and Verifiable Credentials
A decentralized ID (DID) is a new way of managing digital identities that gives control to individuals rather than central organizations (like Facebook or Google). This could be used for secure logins across multiple platforms without relying on centralized entities. DIDs are created and controlled by the person or entity that owns them.
Verifiable Credentials (VCs)—Decentralized Certification Platform: A dApp that issues and verifies educational certificates and diplomas on the blockchain, making it easier to prove and authenticate credentials globally, your VC is linked to your DID]
Bounty track

[Developer tool]

hmm... what is your DID implementation? proposal is kinda vague

Problem
Currently, we use emails and phone numbers as identifiers to access websites and apps, but our access to these identifiers and personal information is at the mercy of service providers, who can revoke it at any time. This means they are centralized. This is where Decentralized Identities (DIDs) come in as a "digital identifier" that anyone can own, independent of any entity, organization, or institution. For example, your email and Facebook account are owned by Google; they can choose to withdraw their service and ban your account for a number of reasons, causing you to lose access to all the websites you signed up for using it.
Solution
DID is a new type of identifier for verifiable self-sovereign identity, i.e., a lifetime, portable, and verifiable digital identity that does not depend on any centralized authority. In other words, DIDs are fully under the control of the DID subject (the person or entity who creates the DIDs), independent from any centralized registry, governance, identity provider, or certificate authority. You have the power to choose who to share it with and revoke access anytime.
Deliverables

User ID Creation and Registration:
Create a system that allows users to generate and register their unique ID on the Kaia blockchain.
DID Wallet:
Create a digital wallet for users to securely store their genrated ID.
Document Storage Functionality:
Add a system so users can store documents (e.g., school certificates, employment records) linked to their ID in the wallet.
Single Sign-On (SSO):
Create a login system to allow Kaia users to log in to third-party websites with their Kaia DID, similar to how they would with an email or Facebook account.

Ok but i need you to be precise on what's the deliverable for this proposal.
The reason why is so the proposal is small enough that matches with the compensation (max 2500 KLAY per submission)
per your response, it seems to be a large system.

Understood. These are two different systems that work together and I have broken it down so they each fit under the max submission requirement.
Deliverable 1: DID Creation Method and Interface
Objective: Develop a method for users to register a decentralized identity on the KAIA network.
Description:  Users access the platform, scan their face using their device's camera, and enter their desired username. The system processes the facial data to create a biometric profile, verifying against existing users. Once verified, users choose a unique username linked to the biometric profile stored securely on their device.
Completion:  Users can successfully create and register their KAIA IDs via the platform.
Deliverable 2: DID Wallet
Tasks: Build a wallet for users to store their generated DID and verifiable credentials (VC).
Key Features:

Secure wallet interface for storing KAIA DID and associated VCs.
User-friendly design for easy access to stored credentials.
Functionality for adding, viewing, and managing verifiable credentials.
Security measures to protect user data within the wallet.

Completion Criteria:  A functional wallet is available for users to store and manage their KAIA DIDs and verifiable credentials.

Ok, approved to proceed. Revert back with your proof of concept


Permission denied after upgrading to<EMAIL_ADDRESS>

Env

Parse-Server 2.2.13
MongoDB 3.0.11
AWS (CentOS + NGINX)

Problem
When trying to delete objects it doesn't work and get Permission denied. After downgrading to 2.2.9, it works fine

looks similar to #2034

Can you try installing 2.2.14, the fix should have been integrated with that change.
Please reopen if the fix is not effective for your use case.

2.2.15 still not working

WHat are the CLP on the class?

Public Read Write

Can you provide the content of the schema for that class?

@benishak?

After enabling "add fields" it worked although it didnt create any new column

hmmm ok... that's odd... Should I close it now? If you want us to keep it open, can you please provide more context (like full schema, CLP, and objects) in order to reproduce and help drill down the error?

It is the standard User class of parse. CLP is public read write delete update

If you want us to keep it open, can you please provide more context (like full schema, CLP, and objects) in order to reproduce and help drill down the error?

Can you provide the actual data? Closing till then.


object/ceph: fix listAll

Codecov Report

Merging #1891 (488d627) into main (0c69891) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1891      +/-   ##
==========================================
+ Coverage   61.29%   61.33%   +0.03%
==========================================
  Files         135      135
  Lines       23411    23411
==========================================
+ Hits        14349    14358       +9
+ Misses       7389     7380       -9
  Partials     1673     1673

Impacted Files
Coverage Δ

pkg/chunk/cached_store.go
75.68% <0.00%> (-0.55%)
:arrow_down:

pkg/chunk/disk_cache.go
79.85% <0.00%> (-0.49%)
:arrow_down:

pkg/meta/sql.go
64.44% <0.00%> (-0.10%)
:arrow_down:

pkg/meta/redis.go
71.39% <0.00%> (-0.05%)
:arrow_down:

pkg/vfs/reader.go
80.29% <0.00%> (+3.10%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c69891...488d627. Read the comment docs.


[Feature] Regular teammates stats

Would be nice to be able to keep track of your regular teammate's stats as well as yours so you can see how the group MMR evolves over time and who's having a hard time or whatever.

I like this idea!
I just pushed an update to the repository with a new tab which tracks the MMR of your most frequent teams and team members.  still a work in progress, but I'll try to get a build of it up soon as well.


Flight history suggestions

I'd like to suggest previous flights as a user types. I had this working at one point but removed it due to a chrome bug.

Cannot do this until this bug is fixed: https://bugs.chromium.org/p/chromium/issues/detail?id=1186804


Improve python detection

New ubuntu versions doesn't have python. It is replaced with python3. Also 2.x is quite old and obsolete. This PR is fixing it. Not tested on mac.

New ubuntu versions doesn't have python. It is replaced with python3. Also 2.x is quite old and obsolete. This PR is fixing it. Not tested on mac.

Yes, agree with moving to Python3, PR under testing now, incl. MAC build. Python related doc should be reviewed and updated as needed.

Tested on MAC as well. Thanks for the effort!


Fix MethodSettings on AWS::Serverless::Api

According to the SAM documentation (and
https://github.com/awslabs/serverless-application-model/blob/master/samtranslator/model/sam_resources.py):

MethodSetting -> MethodSettings
this is an array of MethodSetting, not a single MethodSetting

Thanks!


java.io.FileNotFoundException for old  adam file after upgrade to adam0.20

i have created adam file with old version adam-submit 0.16,
 $docker run -v `pwd`:/data -e SPARK_PRINT_LAUNCH_COMMAND=1 -it heuermh/adam adam-submit count_kmers /tmp/small.adam /tmp/kmers.adam 10

Using ADAM_MAIN=org.bdgenomics.adam.cli.ADAMMain
Using SPARK_SUBMIT=/home/linuxbrew/.linuxbrew/bin/spark-submit
Spark Command: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -cp /home/linuxbrew/.linuxbrew/Cellar/apache-spark/2.0.1/libexec/conf/:/home/linuxbrew/.linuxbrew/Cellar/apache-spark/2.0.1/libexec/jars/* -Xmx1g org.apache.spark.deploy.SparkSubmit --conf spark.kryo.registrator=org.bdgenomics.adam.serialization.ADAMKryoRegistrator --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --class org.bdgenomics.adam.cli.ADAMMain /home/linuxbrew/.linuxbrew/Cellar/adam/0.20.0/libexec/repo/adam_2.11-0.20.0.jar
it will report java.io.FileNotFoundException for  /tmp/small.adam.
i delete the /tmp/small.adam then recreate it with new version adam-submit0.20 , it runs ok

Until we hit ADAM version 1.0 and start maintaining binary compatibility, bdg-formats schema are allowed to change between versions.  Thus an ADAM file written from version 0.16.0 will not necessarily work on version 0.20.0.
As far as the FileNotFoundException, I don't have quite enough here to reproduce.

Hi @Fei-Guang! In your docker command, you don't have a /tmp volume mounted. You just have -v pwd:/data. Perhaps is the file under /data instead of /tmp?


Remote sensor (IT300) for IT500

Hello
Salus IT500 can have 2 zones using a remote sensor (IT300).
Would be possible to control both zones?
Thanks.

Hey @klaudium, unfortunately the person i developed this for does not have the IT300 remote sensors, so i cannot reverse engineer the code required to tap into that portion of the API. If you have any knowledge of reverse engineering API's and can get me the links required to tap into it, feel free to pop it in here and i'll have a look at an implementation.

Closing due to inactivity.


Installation failed on Mac

Hi!
Thank you for the awesome cli command.
I tried to install it on mac using brew but failed.
However, I could install it from source code using cargo.
Thanks in advance.
Brew install errors as follows.
$ brew install greymd/tools/teip
==> Tapping greymd/tools
Cloning into '/usr/local/Homebrew/Library/Taps/greymd/homebrew-tools'...
remote: Enumerating objects: 240, done.
remote: Counting objects: 100% (30/30), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 240 (delta 16), reused 20 (delta 8), pack-reused 210
Receiving objects: 100% (240/240), 1.79 MiB | 3.14 MiB/s, done.
Resolving deltas: 100% (144/144), done.
Error: Invalid formula: /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/tmux-xpanes.rb
tmux-xpanes: Calling bottle :unneeded is disabled! There is no replacement.
Please report this issue to the greymd/tools tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/tmux-xpanes.rb:7

Error: Invalid formula: /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/egzact.rb
egzact: Calling bottle :unneeded is disabled! There is no replacement.
Please report this issue to the greymd/tools tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/egzact.rb:7

Error: Invalid formula: /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/ewscli.rb
ewscli: Calling bottle :unneeded is disabled! There is no replacement.
Please report this issue to the greymd/tools tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/ewscli.rb:8

Error: Invalid formula: /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/echo-meme.rb
echo-meme: Calling bottle :unneeded is disabled! There is no replacement.
Please report this issue to the greymd/tools tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/echo-meme.rb:9

Error: Invalid formula: /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/issagen.rb
issagen: Calling bottle :unneeded is disabled! There is no replacement.
Please report this issue to the greymd/tools tap (not Homebrew/brew or Homebrew/core):
  /usr/local/Homebrew/Library/Taps/greymd/homebrew-tools/issagen.rb:7

Error: Cannot tap greymd/tools: invalid syntax in tap!

My environment

System Software Overview:

  System Version:	macOS 12.3.1 (21E258)
  Kernel Version:	Darwin 21.4.0
  Boot Volume:	Macintosh HD
  Boot Mode:	Normal
  Secure Virtual Memory:	Enabled
  System Integrity Protection:	Enabled

Hardware Overview:

  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro16,2
  Processor Name:	Quad-Core Intel Core i5
  Processor Speed:	2 GHz
  Number of Processors:	1
  Total Number of Cores:	4
  L2 Cache (per Core):	512 KB
  L3 Cache:	6 MB
  Hyper-Threading Technology:	Enabled
  Memory:	16 GB
  System Firmware Version:	17<IP_ADDRESS>.0 (iBridge: 19.16.14243.0.0,0)
  OS Loader Version:	540.100.7~23
  Activation Lock Status:	Disabled

Hi,
I have just update the formula of syntax to fix this issue. Could you confirm if it works fine now?

It's now working as expected!
Thank you for fixing quickly


ability to change vocoder

is it possible to integrate other models such as HiFi-Gan?

Yes, I add this feature just now. Check out the readme: https://github.com/haoheliu/voicefixer#others

Thank you with your efforts!


add rule (rrd) comparison to 0

if (results.length > 0) {

should be
if (results.length) {

@David-Pena there is somebody who might do this, so please do not start it yet.

He is working om #143  so this is yours.

I created a PR for this #159


Unstable Affliction

Currently only shows 5 ticks instead of 6 with the new 18s duration.
Also is it possible to add the dispel dmg for it? like how the initial dmg on immolate works along with the dot

I can fix the duration but atm I don't have a great way of displaying dispel dmg without it being included in total dps and evaluation which I don't think it should be so I'll leave it as it is.


pkg/dartdev/test/commands/run_test flaky -> RuntimeError

There are new test failures on [dds] Add DAP support for 'hotReload' custom request that calls reloadSources.
The tests
pkg/dartdev/test/commands/run_test RuntimeError (expected Pass)

are failing on configurations
unittest-asserts-release-win

/cc @DanTup

@bkonyi it looks like these failures are in run_test.dart:
https://dart-ci.firebaseapp.com/#showLatestFailures=true&configurations=unittest-asserts-release-win
I don't think any code in the CL above should be running for those tests at all - could these failures be unrelated flakes?

It looks like this is still an issue, even after https://github.com/dart-lang/sdk/commit/2408d388782826227a32df5e59a769101819edca. @athomas @whesse, do either of you have any other suggestions on how to resolve this?
I'm going to mark this test as approved again in the meantime.

I think the test could catch the exception thrown during the deletion of the temp directory, and ignore it.  Failing to delete a temp directory created for a test is not a failure of the test, and should not be a fatal infrastructure failure, I think.
On our CI bots, the directories might be kept by a local cache of that builder directory, which is reused, but the builders get recycled within a day on linux and on windows.  It would be nice for every test to always successfully clean up its working directory, but I think we can allow an occasional failure to do it, which is not registered as a test failure or an infra failure.

I wasn't clear enough in my first comment in the gardening doc, reposting the last comment on that thread:

I think you need Process.start, because with run you also don't get the pid to kill in the teardown. If those processes start other processes there might be a need to track those and kill 'em too (one way would be by using OS specific ways to kill process trees).

Just using run instead of runSync doesn't make the test stop leaking processes. Leaving the temp directories around will turn the build purple (because there will still be a process preventing the deletion when the infra tries to clean up temp directories).
The solution is to use Process.start, track all started processes and kill them in the tear down (or when the test timed out). The test is leaking processes, that's the real problem.

I wasn't clear enough in my first comment in the gardening doc, reposting the last comment on that thread:

I think you need Process.start, because with run you also don't get the pid to kill in the teardown. If those processes start other processes there might be a need to track those and kill 'em too (one way would be by using OS specific ways to kill process trees).

Just using run instead of runSync doesn't make the test stop leaking processes. Leaving the temp directories around will turn the build purple (because there will still be a process preventing the deletion when the infra tries to clean up temp directories).
The solution is to use Process.start, track all started processes and kill them in the tear down (or when the test timed out). The test is leaking processes, that's the real problem.

I'm using Process.start and killing the process on tear down in 2408d38 :-) However, that still doesn't seem to fix this issue, hence the followup CL to catch the exception on Windows.

@bkonyi Sorry, I should've looked closer... 🤦
I wonder if the problem is in dispose():
  void dispose() {
    _process?.kill(); // <-- doesn't block until the process is dead.
    _process = null;
    if (dir.existsSync()) {
      dir.deleteSync(recursive: true); // <-- process might still be running and locking the directory.
    }
  }

So we probably need to await the exit code before deleting the directory.

This test starts failing on pkg-win-release:
01:12 +17 ~1 -1: run DDS explicit enable dart run simple [E]

  FileSystemException: Deletion failed, path = 'C:\b\s\w\ir\x\t\a4ba6124f' (OS Error: The process cannot access the file because it is being used by another process.

  , errno = 32)

  dart:io/directory_impl.dart 201:7                 _Directory._deleteSync
  dart:io/file_system_entity.dart 420:7             FileSystemEntity.deleteSync
  pkg\dartdev\test\utils.dart 100:11                TestProject.dispose
  pkg\dartdev\test\commands\run_test.dart 26:21     run.<fn>
  dart:async/future.dart 296:31                     new Future.sync
  package:test_api/src/backend/invoker.dart 236:18  Invoker.runTearDowns.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 257:15  Invoker._waitForOutstandingCallbacks.<fn>
  package:test_api/src/backend/invoker.dart 254:14  Invoker._waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1426:13                      _rootRun
  dart:async/zone.dart 1328:19                      _CustomZone.run
  dart:async/zone.dart 1861:10                      _runZoned
  dart:async/zone.dart 1783:10                      runZoned
  package:test_api/src/backend/invoker.dart 254:5   Invoker._waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 235:9   Invoker.runTearDowns.<fn>
  package:test_api/src/backend/invoker.dart 230:21  Invoker.runTearDowns.<fn>
  dart:async/zone.dart 1426:13                      _rootRun
  dart:async/zone.dart 1328:19                      _CustomZone.run
  dart:async/zone.dart 1861:10                      _runZoned
  dart:async/zone.dart 1783:10                      runZoned
  package:test_api/src/backend/invoker.dart 230:12  Invoker.runTearDowns
  package:test_api/src/backend/invoker.dart 398:52  Invoker._onRun.<fn>.<fn>.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 257:15  Invoker._waitForOutstandingCallbacks.<fn>
  package:test_api/src/backend/invoker.dart 254:14  Invoker._waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1426:13                      _rootRun
  dart:async/zone.dart 1328:19                      _CustomZone.run
  dart:async/zone.dart 1861:10                      _runZoned
  dart:async/zone.dart 1783:10                      runZoned
  package:test_api/src/backend/invoker.dart 254:5   Invoker._waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 398:17  Invoker._onRun.<fn>.<fn>.<fn>

Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.

https://dart-ci.appspot.com/log/pkg-win-release/unittest-asserts-release-win/18913/pkg/dartdev/test/commands/run_test

Started failing again.
01:24 +20 ~1 -1: run DevTools dart simple [E]

  FileSystemException: Deletion failed, path = 'C:\b\s\w\ir\x\t\af619c7b7' (OS Error: The process cannot access the file because it is being used by another process.

  , errno = 32)

  dart:io/directory_impl.dart 201:7              _Directory._deleteSync
  dart:io/file_system_entity.dart 420:7          FileSystemEntity.deleteSync
  pkg\dartdev\test\utils.dart 101:11             TestProject.dispose
  ===== asynchronous gap ===========================
  pkg\dartdev\test\commands\run_test.dart 26:24  run.<fn>

https://dart-ci.appspot.com/log/pkg-win-release/unittest-asserts-release-win/18983/pkg/dartdev/test/commands/run_test
I don't see how my change could cause this error.

Failed once again:
01:40 +21 ~1 -3: run DevTools dart run disabled [E]

  FileSystemException: Deletion failed, path = 'C:\b\s\w\ir\x\t\acf7602be' (OS Error: The process cannot access the file because it is being used by another process.

  , errno = 32)

  dart:io/directory_impl.dart 201:7              _Directory._deleteSync
  dart:io/file_system_entity.dart 420:7          FileSystemEntity.deleteSync
  pkg\dartdev\test\utils.dart 101:11             TestProject.dispose
  ===== asynchronous gap ===========================
  pkg\dartdev\test\commands\run_test.dart 26:24  run.<fn>

https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8827020818389088193/+/u/test_results/new_test_failures__logs_

and again https://dart-ci.appspot.com/log/pkg-win-release/unittest-asserts-release-win/19018/pkg/dartdev/test/commands/run_test

@athomas @whesse, it looks like we're still having issues here, even after waiting on the process exit code. Should we maybe consider catching the exception since this isn't a symptom of an actual test failure?

and again on https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8825685537532347809/+/u/test_results/new_test_failures__logs_

Got https://dart-review.googlesource.com/c/sdk/+/226980 with the "fix". I am able to reproduce issue locally and confirm that cl addresses the problem.

The fix made another test change from Pass to flaky when it landed, and now to consistently failing with RuntimeError:
dartdev/test/analysis_server_test on Windows.
https://dart-ci.appspot.com/log/pkg-win-release/unittest-asserts-release-win/19284/pkg/dartdev/test/analysis_server_test

Thanks @whesse . I think that should be logged as a new separate issue.


Design Commons Help & Support page for clearing HC cookies

Based on this article, which is regularly sent to users with login issues and typically sent via<EMAIL_ADDRESS>

Created page on help site: https://support.hcommons.org/i-am-having-trouble-logging-in-what-should-i-do/


ion-datetime scroll is very slow

Short description of the problem:
When using ion-datetime scroll is very slow especially, on Android
Steps to reproduce:

Use ion-datetime or the plunker below
Scroll
Look at logs

It's worse on Android because of Cordova logging to ADB on each console.log.
When looking at the console when can see the component logging at each movement, here is the responsible code at src/components/picker/picker-component.ts@l233
console.log(`decelerate y: ${y}, velocity: ${this.velocity}, optHeight: ${this.optHeight}`);

Which Ionic Version? 2.x
Example of the issue
http://plnkr.co/edit/7gBqdyNSQJWdfPNws2bq?p=preview

ionic info:
Cordova CLI: Not installed
Ionic Framework Version: 2.0.0-rc.0
Ionic CLI Version: 2.1.0
Ionic App Lib Version: 2.0.0-beta.20
ios-deploy version: Not installed
ios-sim version: Not installed
OS: Mac OS X El Capitan
Node Version: v6.7.0
Xcode version: Xcode 8.0 Build version 8A218a

seems to be the same problem of #6613
I used the https://ionicframework.com/docs/v2/native/Date Picker/ instead of it. It has a better ux.

I don't think it the same issue, the problem is not about the virtual scroll continuing to scroll but the performance issues due too the excessive logging. I already use native picker on iOS and Android but I'm using Ionic on browser too. If I disable logging as mentioned in the pull request it works pretty well.

Fixed by https://github.com/driftyco/ionic/commit/713e2a1a33ded9dd238a3babe16db72c2f751741
Please mark issue as resolved


Supporting RWKV models

🚀 The feature, motivation and pitch
Linear attention allows for longer context and faster inference. The eagle model has a 2T checkpoint soon.
Alternatives
NA
Additional context
No response

Hello, are you working on adapting vLLM for RWKV?

https://github.com/harrisonvanderbyl/vllm Having a go at it here

Added in #11193


React Native

Hi,
Does this plugin work with React Native?

This plugin and Rete.js core are based on DOM, so you need to to embed this through Webview to your RN app


Can't use ActiveRecord scopes directly

With a model like this:
# app/models/child.rb
class Child < ActiveRecord::Base
  validates_presence_of :name
  def self.not_null # contrived example
    where("name is not null")
  end
end

And a template like this:
# index.json.json_builder
children Child.not_null do |c|
  id c.id
end

Rails throws:
undefined method `id' for #<ActiveRecord::Relation:0x0000000548ca60>

This is fixed by adding .all to the where call (i.e. where("name is not null").all), but that would break chained scopes (i.e. Child.not_null.order_by_name), so it's not a solution, or by adding .all when calling the scope (i.e. Child.not_null.all), but it shouldn't be necessary.
I believe this error is line 13 in Member which asks if argument.is_a?(Array), to which an ActiveRecord::Relation returns false.
I guess the solution would be to check whether argument is "array-like" rather than an actual array (maybe something like if argument.respond_to? :each, but that would catch Hashes and anything else that has an each method), or adding || argument.is_a?(ActiveRecord::Relation) (but that would be a Rails-specific solution).

I'd like to hear if you have anything against this. Is checking explicitly for an Array and ActiveRecord::Relation too narrow? Would explicitly leaving out Hash been too open?

Yeah, I was thinking of a solution like that; I just didn't know if you wanted to have ActiveRecord-specific code in the library (for example, maybe this code won't work with another ORM).
Actually, at first I tried using the array method, which worked fine, except that by using it I can't include anything else (this had me scratching my head for a while, by the way). I think array is nice, because you are explicitly telling the builder to treat the object as an array, so the user can throw whatever he wants in there and it will work as long as it responds to each.
So my idea is: maybe add an option to the array method accepting a custom key?
Something like:
array @child, key: :children do |child|
  id child.id
end

Or you could even forego "key" and go with array @child, :children.
What do you think?

Per the README, the array method will build an array with JSON objects, where each object defines their own keys.
What I think you're trying to do is:
children @child do |child|
  id child.id
end

JSON Builder tries to be smart and as you can tell, checks to see if the passed object is an array, if so, create an array member and return it as such. But if you want to create an array without a key attached to it, that's when you'd use the array method.

(PS: I edited my previous comment (@child => @children) to make it clear that the object represents "many children".)
Yes, that's what I understood from reading the README. That's when I replaced "array" with "children", and it didn't work because my array wasn't an array but an ActiveRecord::Relation.
Your commit will work fine for me and most people I think, but it will break when the object is not an array nor an ActiveRecord::Relation, but some other array-like object.
So I had another idea:
How about checking if a block is given, and if the .arity of the block is 1, then build an array and treat the object as an array?

Oh ok, I thought that comment was a separate issue that wasn't really related to the first.
If you'd like to provide a pull request with that I'd be more than happy to merge it in, but for right now I feel it is handling the arrays for a large number of people correctly. Could you see of any other situation where passing in anything but an Array or query/relation may need the actual array block?

Yes, the way it is now is much simpler, and it probably solves the problem for 95% of the use cases. Plus, I really can't think of an array-like object that's not an array/enumerable nor an ActiveRecord::Relation; my "arity" idea was to future-proof the code =)
For now, I guess it's not really necessary, but I'll try to make a patch just for fun.


Correct path + remove unneeded graphql import

Removing passing in path surfaced a bug in the data where one of the paths was missing its forward slash. Gatsby adds to paths if you don't add it but because of the discrepancy, the query now failed because the path in the page object was different than the path in the markdown file.

For this reason and others, I recommend always just using the ID to select things. IDs are guaranteed to be unique across the site.

Hey Kyle,
Apologies if I'm missing something obvious, but why are the graphql imports unneeded?

The graphql template tags aren't executed at run-time — they're extracted at build-time. We use them the same way as Relay https://facebook.github.io/relay/docs/en/graphql-in-relay.html#graphql
Also, graphql, the npm package is for running backend graphql servers


v1.5 crashes when parsing the latest apnic data

Running geofeed-finder-linux-x64-1.5 -i apnic or without "-i apnic" crashes. The last two lines of output:
inetnum: <IP_ADDRESS>/23 https://ns1.vpshispeed.com/geofeed.csv [cache]
Illegal argument undefined

The resulting "result.csv" is not created. A week ago it was running fine. The crash is caused by the new apnic data. Deleting the cache doesn't help.

Hi @sgteq,
Thanks a lot for reporting this!
To improve performance, I was first filtering remarks starting with the word Geofeed and after properly validating the associated url.  However, I was not handling properly the case when somebody adds a remark Geofeed  not followed by the url. This happened 3 days ago with an inetnum in apnic.
This is now fixed and released in v1.5.1. Sorry for the inconvenience.

Thanks for the quick fix @massimocandela ! The issue is fixed indeed but I discovered a side effect of v1.5.1 release -- it finds 53 fewer feed URLs, a significant drop.
all-2022-08-30.log: 299 URLs 4,166,912 IPv4 addresses
all-2022-09-04.log: 252 URLs 3,546,950 IPv4 addresses

53 previous URLs not found, 6 new URLs found = net -47 URLs. It appears due to multiple reasons. One of the reasons is that v1.5.1 is more strict about remark format. But another is unclear. I'm submitting it as a separate issue.


Component is not displayed until Refresh clicked on command menu

When the ModelDrivenGrid solution is published and executed in PowerApps, nothing is displayed until Refresh is clicked on the command menu.
Maybe this is by design?
If not, this can be fixed by inserting ...
this.context.parameters.records.refresh();
... at the end of the public init method.

@LozzieDuff Are you asking about one of our code samples? If so, which one?

@phecke Hi there, yes, the ModelDrivenGridControl ...
https://github.com/microsoft/PowerApps-Samples/tree/master/component-framework/ModelDrivenGridControl
I downloaded and installed it as is (great demonstrator, btw), but it didn't appear to show any data.  Hitting browser refresh didn't help.  Hit the command bar Refresh and whoosh, there it was.  Stepped through the code (I'm a newbie with the JSX, React, PCF frameworks/stacks/whatever(!), more dotnet and vanilla JavaScript experience) but eventually took a punt on adding -- this.context.parameters.records.refresh(); -- at the end of the public init method.  That worked.
Whether it is the right thing to do or if I've missed a bit out of newbie-ness I don't know!
If it is a valid "fix", can I have a go at branching and pull request the "fix" in and all that jazz?  My first contribution - break my Github virginity, so-to-speak!

For me the component is loading but data isn't, but when I click on "Refresh" in the command bar it starts showing data. Even clicking on sorting shows the data.

During page load

After clicking "Refresh" or "Sorting"

Yep, that's the issue I experienced.
As said, try adding: this.context.parameters.records.refresh(); -- at the end of the public init method.

Yes that's correct.

@LozzieDuff
Thanks. I've created a workitem to review this tutorial with focus on those steps.
3037743


Waiting for code...

Waiting for code...

Waiting for code...

Dear all, thanks for your interest in our paper and thanks for your patience. We finished reorganizing the code before the end of 2022 and published it on GitHub.
And we are sorry that we mistakenly place the code in the 'dev' branch that did not display on Github's webpage by default. Now we have fixed this issue and please see the code at https://github.com/Thinklab-SJTU/GAMF/tree/dev


Print out all of the MISC_INFO fields

This includes some additional extensions to the macros that manipulate MISC_INFO fields to handle versions and simple printing cases. misc_accessors! has been changed to always yield an Option, and always yield references.
Yielding references avoids very large copies for the more enormous fields, at the cost of making it slightly more awkward to use the smaller POD values. Adding special casing for reference-vs-value seemed like a needless mess, but it can be done if desired.
Making everything an Option is the result of adding support for fields which e.g. don't exist in MISC_INFO_2 but do exist in MISC_INFO_3. This is an "unnecessary" regression for precisely the first two fields of MISC_INFO which are always guaranteed to exist. Adding special casing for those two seemed like a needless mess, but it can be done if desired.
This builds on #137 (commit included until it lands).
I am a bit uncertain about how I handle SYSTEMTIME instances here. They don't seem to be "normal" SYSTEMTIME's (e.g. in the one minidump I'm testing with the year is 0). Also I just drop the day-of-the-week entry since that seems uninteresting.

Did some cleanup on this to fixup most of my annoyances with my first pass. Initial comment updated to reflect the current state.


stylechecking

This has been done via autopep8, but I've also manually verified it.

Postponing a bit to merge other PRs, otherwise we get a lot of merge conflicts.

So many merge conflicts... Better to do such an effort in small steps.


added four blog resources

What does this PR do?
Includes these blog resources:

Chrome Developers Blog
IBM's Data and AI Blogs
Wesbos's Blog
Wake Up And Code

Thanks!

Thanks!

Welcome :smile:


suppert type of actino.type is Symbol.

when my action.type is Symbol, action = {type: Symbol("CALL HTTP API")}.
redux-logger throw an error:  Cannot convert a Symbol value to a number.
so I fix it.

It's unclear to me how this error might occur in the line that you've changed. Are you sure that it was caught right at that point?
We actually have an example of an action constant created with the help of Symbol and it works properly. You can create such an action here by clicking on "UPDATE TOKEN" button.

Transform Symbol() action type to string.

@toxickg I don't know why your code work well, because the code comporessed.
and I can't run example In this repository, because a dependecy @react-ui/core is not in the npm registry.
and here is a very simple test code.
Symbol("call api") + "ddf", you can run this code in chrome devtools, and it will throw an error
Cannot convert a Symbol value to a number.

@toxickg @theaqua I find the problem,
const store = createStore(rootReducer, applyMiddleware(thunk, ...middlewares, createLogger()))
I use createLogger without option.


Delete roles on /unkusss

Is your feature request related to a problem? Please describe.
Currently only the permissions are removed. This means that the guild still saves the channels if they are empty.
Describe the solution you'd like
Kilian should automatically remove channels if no one has permission to it any more and additionally they are empty. If they are not empty they should get moved to an archive. If someone joins again the channel should be retrieved from the archive again.

Already finished, will be available with archive


Revert "Change the type of Samza Runner MaxBundleSize pipeline option (#25854)"

This reverts commit 14ca840f3462aa9ba3ebbe773b498d4a914d50aa.
We found out there are other problems when having both Samza and Flink runners in the same classpath, so this change doesn't help but introduces some backward incompatible side effects. Reverting it for now.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java PreCommit


Add abstain rev acct

Add abstain rev acct for when someone changes their vote to abstain.
does not vote abstain unless vote is changed back to abstain so total abstain votes cannot be counted and may not be needed. support and oppose can be counted and the rest assumed to be abstain.

I amended dfb2296 to take out the change to package.json and then merged it: c439017870c7

the package.json references yalc, we did not need that

the package.json references yalc, we did not need that


Semantics of ArrayView and const

Should we be able to modify the data within a const ArrayView?
We should probably follow the semantics of std::span

This was discussed here: https://github.com/LLNL/axom/issues/633#issuecomment-964546470

Let's look at what RAJA Views are doing as well.

Here are some bread crumbs for what we do in RAJA. Note that this looks more complicated than it needs to be. The reason is that this stuff is used to make view data access, via an operator that takes a tuple of indices based on dimensionality of the view (e.g., view(i, j, k)), generic. That is, indices can be C++ language defined scalar integral types, strongly typed (e.g., i, j, k are distinct types to prevent indexing errors at compile time), or types used for SIMD vector, matrix, tensor based iterates.
Here's the generic operator (i.e., view(i,j,k)) definition: https://github.com/LLNL/RAJA/blob/develop/include/RAJA/util/TypedViewBase.hpp#L334
Here's the definition of the method to create the view return value: https://github.com/LLNL/RAJA/blob/develop/include/RAJA/util/TypedViewBase.hpp#L151
Other relevant definitions for 'ViewReturnHelper', etc. are in the RAJA/include/util/TypedViewBase.hpp file.


Cards: Update design-system to 0.1.0

Changes

Updates design-system to 0.1.0
Removes card component less files from cfgov-refresh, which are delivered through cfpb-layout.

Testing

Pull branch and run ./frontend.sh.
Cards one homepage and on http://localhost:8000/ask-cfpb should be unchanged from production.

@wpears Maybe we could put some instructions in the wiki or something on how to gracefully update packages? Not sure what I'm doing wrong.

Maybe we could put some instructions in the wiki or something on how to gracefully update packages?

Yes plz. I think most people edit .yarnrc, update packages and then reset .yarnrc.

Yes plz. I think most people edit .yarnrc, update packages and then reset .yarnrc. I'm not sure what the proper process is.

I use yarn-check for the updates, which obviously misses a step.

Maybe we could put some instructions in the wiki or something on how to gracefully update packages?

Yes plz. I think most people edit .yarnrc, update packages and then reset .yarnrc.

That should definitely not be necessary, right?
If memory serves, doesn't it come down to needing to use yard add package for everything and not manually updating package.json?

If memory serves, doesn't it come down to needing to use yard add package for everything and not manually updating package.json?

Ah, cool, I remember it being flaky but I guess that's not the case anymore.

I've been testing this a bit, I think the truly foolproof command is:
yarn cache clean && yarn add<EMAIL_ADDRESS>I think if you have a package in your global yarn cache you won't get one generated in the yarn offline cache local to the project. This is imo a completely backwards design decision which I still need to validate. If it is indeed true, then I was thinking of just adding a small command to the package.json to smooth this a bit.

Just added a few docs in dev tips: https://github.com/cfpb/cfgov-refresh/pull/5632

Thanks @wpears !

Review plz!


GraphQLError: Names must only contain [_a-zA-Z0-9] but "application/pdf" does not

Hey 😊
I encountered this error:
GraphQLError: Names must only contain [_a-zA-Z0-9] but "application/pdf" does not.
      at assertName (/platformatic/node_modules/.pnpm/graphql@16.6.0/node_modules/graphql/type/assertName.js:31:13)
      at assertEnumValueName (/platformatic/node_modules/.pnpm/graphql@16.6.0/node_modules/graphql/type/assertName.js:58:10)
      at /platformatic/node_modules/.pnpm/graphql@16.6.0/node_modules/graphql/type/definition.js:1223:49
      at Array.map (<anonymous>)
      at defineEnumValues (/platformatic/node_modules/.pnpm/graphql@16.6.0/node_modules/graphql/type/definition.js:1213:35)
      at new GraphQLEnumType (/platformatic/node_modules/.pnpm/graphql@16.6.0/node_modules/graphql/type/definition.js:1080:20)
      at constructGraph (/platformatic/packages/sql-graphql/lib/entity-to-type.js:58:21)
      at mapperToGraphql (/platformatic/packages/sql-graphql/index.js:43:20)
      at Plugin.exec (/platformatic/node_modules/.pnpm/avvio@8.2.0/node_modules/avvio/plugin.js:130:19)
      at Boot.loadPlugin (/platformatic/node_modules/.pnpm/avvio@8.2.0/node_modules/avvio/plugin.js:272:10) {
    path: undefined,
    locations: undefined,
    extensions: [Object: null prototype] {}

To add a little more context, the table where it failed contained a column defined like that:
`document` enum('application/pdf','image/jpeg','image/png') DEFAULT NULL,

Can we support this column type?
In the meanwhile, I'll add a PR to enhance the logging info on platformatic side when we encounter this.

Agreed! I think some transformation of names is required for this case.

Would you like to send a PR to fix this?

Yup, I'll try that ;)

I think this might be the same of https://github.com/platformatic/platformatic/pull/403.

Done on https://github.com/platformatic/platformatic/pull/516

Fixed in https://github.com/platformatic/platformatic/pull/516


Use of tensorflow-GPU

Do you have any recommendations on the use of tensorflow-gpu?
Description
I am currently attempting to run AutoKeras in my institution's HPC, but it is highly encouraged to use tensorflor-gpu in order to correctly use the gpu. As you know, autokeras does not include tensorflow-gpu. Will having two versions of tensorflow have any effect on autokeras? Will it create a conflict?
Current tensorflow
tensorflow=1.10.0
Alternative Solutions
Install the matching version of tensorflow-gpu
pip install tensorflow-gpu=1.10.0

Upgrade autokeras, now it supporta GPU.


Add RISC-V bit manipulation instruction 'sbinv'

funct7 | rs2 | rs1 | f3 | rd | opcode
| 0110100 | rs2 | rs1 | 001 | rd | 0110011 |

Thanks, but where are unit tests?

Ninja doesnt work on my PC, i did everything like in instruction, but it stil cant find build.ninja file (google didnt help), so i wanted to ask about that tomorrow, if it allowed of course. I really dont know what is going on with this program

Hope Kirill answered your questions. If not, please don't hesitate and e-mail me or him.


Running carbonzipper with carbonserver on custom ports not working ?

Carbonzipper and carbonserver from latest master.
I would like to test this setup next to standard graphite-web setup, but i have some problems.
2015/08/30 13:27:14 starting carbonzipper (development version)
2015/08/30 13:27:14 setting GOMAXPROCS= 2
2015/08/30 13:27:14 Using graphite host loadbalancer-graphite-instances:2013
2015/08/30 13:27:14 listening on :8080
2015/08/30 13:27:14 singleGet: error querying  <IP_ADDRESS>:8080 / /metrics/find/?format=protobuf&query=%2A : unsupported protocol scheme ""
2015/08/30 13:33:04 singleGet: error querying  <IP_ADDRESS>:8080 / /render/?format=protobuf&from=1440940588&now=1440941188&target=diamond.fluentd.%2A.%2A.processcpu.fluentd&until=1440941188 : unsupported protocol scheme ""
2015/08/30 13:33:04 render: error querying backends for: /render/?target=diamond.fluentd.*.*.processcpu.fluentd&format=pickle&from=1440940588&until=1440941188&now=1440941188 backends: [<IP_ADDRESS>:8080]

Config for carbonzipper running on graphite-web/api instances:
{
  "Backends": [
    "<IP_ADDRESS>:8080"
  ],
  "GraphiteHost": "loadbalancer-graphite-instances:2013",
  "MaxProcs": 2,
  "Port":     8080,
  "Buckets":  10,
  "TimeoutMs": 10000,
  "TimeoutMsAfterAllStarted": 2000,

  "MaxIdleConnsPerHost": 100
}

Communication is working to carbonserver from carbonzipper hosts:
telnet <IP_ADDRESS> 8080
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.

And nothing in carbonserver logs:
2015/08/30 13:12:21 starting carbonserver (development build)
2015/08/30 13:12:21 reading whisper files from: /opt/graphite/storage/whisper
2015/08/30 13:12:21 maximum brace expansion set to: 10
2015/08/30 13:12:21 set GOMAXPROCS=2
2015/08/30 13:12:21 listening on :8080

Carbonserver running with such parameters on graphite instances with whipser files - different instances then carbonzipper:
/usr/bin/carbonserver -logdir=/var/log/carbonserver/ -maxexpand=10 -maxprocs=2 -p=8080 -scanfreq=0 -w=/opt/graphite/storage/whisper -stdout=true -vv=true

From what i was able to get from code, configs should looks like i describe. What is wrong with that setup ? Any suggestions ?

Need actually to prefix the URLS with http://. Docs are lacking. Sorry :(

Hello, ive been using carbon-zipper for almost a week now, taking a mirror or all prod traffic, and it looks to be working very well, so many thanks :) next week ill start trying to break things.
anyway, as per these comments, would it be possible to get a list of arguments that the app takes?
ive only used:
-c=./zipper.json -p=8080 -stdout -d=3
even if its just a quick list it would be a great help to me, many thanks.

@dgryski thank, yes with http it is working now. Using curl all looks to work well, but i have no time to check if graphite-web will work with single local metrics endpoint.

The only command line flag you're missing is -logdir, which sets the output directory for the log files.
Also interesting are the values available to be set in the config file, which I've annotated with some comments:
var Config = struct {
	Backends []string
	MaxProcs int   // GOMAXPROCS
	Port     int        // port to listen on
	Buckets  int    // number of buckets for timings aggregations

	TimeoutMs                int        // timeout to wait for a response from the backend
	TimeoutMsAfterAllStarted int  // timeout to wait after all goroutines to the remote backends for a single metric have started

	GraphiteHost string  // the destination host to send graphite metrics to

	MaxIdleConnsPerHost int // number of idle connections to keep open to carbon backends

	ConcurrencyLimitPerServer int  // maximum number of concurrent requests for a single server

@szibis graphite does work with a single local backend ,but it needs to be patched to accept localhost.  You need the tiny patch listed on http://www.iamsysadmin.ninja/2014/12/graphite-scaling-and-my-evaluation-of.html

Closing.  But opening another bug to improve documentation.


xTeve opens to configuration screen

I am running the latest version of xTeve and have a backup instance of it running on another computer.  I am using it to feed Plex.  Once computer is running W10 and the other W11. I have tried running it as a service NSSM and just in the Startup folder.  I spend the time building out all the mappings and things work great for about a week, then out of the blue it resets.  When I login to the web portal I am presented with the Configuration screen and cannot move past it without starting as a new server.
This happens on both my main server and my backup server.
This started happening when I installed it via NSSM.

Our Dev has recommended NSSM, but I have always used Task Scheduler and a simple .bat file. We can't support NSSM since it's a  program ran by others.

I have no idea since I don't use NSSM.


Hide/Show cell phone field based on country code

What's this PR do?
This PR hides the cell phone field if the users country code is not in our country language map, otherwise for US, MX, BR, CA and UK we show and collect that info.
Where should the reviewer start?
The magic happens in register.inc
How should this be manually tested?
Change your country code or use VPN to pretend you are in another country and when you try to create an account you should only see the cell phone field if you are in US, MA, BR, CA, or UK.
Any background context you want to provide?
One thing, I sorta lied. Right now, CA and UK are not in our language map so the cell phone field will be hidden for those counties. However, #5205 will add those countries to keep me honest.
What are the relevant tickets?
Fixes #5085

@angaither updated all the things.

:+1:


[BEAM-4640] Samza runner postcommit ValidatesRunner job

Please add a meaningful description for your change here

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Spark

Go

---
---
---
---
---

Java

Python

---

---
---
---

@kennknowles : please take a look. Thanks!

I also noticed that gradle :beam-runners-samza:validatesRunner executes no tests. I looked around but couldn't figure out exactly why. Can you see about that?

I just ran ./gradlew :beam-runners-samza:validatesRunner in my box and it ran 208 tests in the end. Is there anything you saw different in jenkins?

Really curious. When I run ./gradlew :beam-runners-samza:validatesRunner on my box there are no tests run. Well anyhow having the Jenkins job is the right start.


Upgrade the npm package of watson-developer-cloud into 2.9.0

In order to use the new translator service in watson cloud, we need to
change the version of the npm package in the dockerfile.

This PR works together with the PR in openwhisk-catalog: https://github.com/openwhisk/openwhisk-catalog/pull/153.

@houshengbo can you run a pg build for this please?

running pg2#631

@houshengbo this page https://github.com/openwhisk/openwhisk/blob/master/docs/reference.md should also be updated


Intermittant raised exception class 201

function canMove in map.pas occasionally raises an exception when checking if an entity can move

Looks like this has been fixed since tidying up the code. A closing bracket was in the wrong place.

This issue is still ongoing

probably fixed...
I've added a debugging option that prints the random seed to the console when the game is run. If the issue appears again it can be debugged using the same seed to recreate the crash


DBBeaver Connects to an older version of database

In MSSQL Server, I connect to my local DB MyDB using following connection settings:
Server Name: (localdb)\MSSQLLocalDB
, that works fine, I check a Table-A which has some records I added, all fine and working.
In DBBeaver, to connect to same local DB MyDB, I use following:
Host: localhost
Database/Schema: MyDB

, that connects fine and all is working, too.  However, this shows an older version of Table-A.  I know that because I changed a column name in this table from COL-1 to COLUMN-A and I added couple of records in it
So, on my SQLServer, Table-A has following
COLUMN-A
Record 1
Record 2

But on my DBBeaver, Table-A has following
COL-1
No records

So, it looks like while my SQLServer is connecting to the very latest version of MyDB database, DBBeaver is somehow connecting to an older version of MyDB with an old column name Col1 which has no records in it.

Most likely these are completely different databases.
From DBeaver you are connecting to the local SQL Server instance, not LocalDB.
See this: https://github.com/Microsoft/mssql-jdbc/issues/769
Looks like Microsoft driver doesn't support LocalDB at all.
jTDS driver may work but, honestly, I didn't test this.

There is no update almost a year.
Ticket is closed.


requireDualQuit button logic is no longer working on v1.0.1 builds

when triggering the allowedDeferralsUntilForcedSecondaryQuitButton code base, it is no longer working with some of the refactoring.

Confirmed with user that it's fixed.


cyrus_sasl: fix GSS-SPNEGO not detected with autoconf 2.71

Motivation for this change
Fixes https://github.com/NixOS/nixpkgs/issues/157803
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/743

Is this a 21.11 only issue?

it will be fixed on unstable by https://github.com/NixOS/nixpkgs/pull/161417


S3 object keys are not quoted

(At least) S3Client.get_object fails to quote the object key into the URL it generates for the request.  For any key with anything that's a url special character, this causes problems.
AWS maybe respond to such requests with a weird looking 505 claiming an unsupported HTTP version was requested.

This was fixed somewhere along the way. Except, it was also broken differently at the same time: signing ends up using a double-encoded path and fails.


heat map plot returns runtime error on MacOS

code to reproduce:
def foo:
    l2 = [12,16,13,14,15]
    l1 = [1,2,3,4,5]
    col2 = Column.fromList "col2" Int64Type l2
    col1 = Column . fromList "col1" Int64Type l1
    fromColumns1 = Table.fromColumns [col1 , col2]
    correlations1 = fromColumns1 . correlations
    heatmap1 = correlations1 . heatmap
    None

reproducable on windows as well


Fix critical error in flowlogs pkg

Issue #, if available: 1931
Description of changes:
https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/ec2#DescribeFlowLogsOutput
FlowLogID is not a field defined in DescribeFlowlogsOutput. Need to go inside the resp.FlowLog[] to get the FlowLogID. This is causing a crash any time FlowLog resource is created because it is always going to return index out of bounds.

I realize my code may not be exactly right, I am just basing this off my knowledge of the sdk.

@mattzech I just realized that you are pointing to an sdk v2 link, while in ACK we are using the v1

@a-hilaly Ah, that would make sense as to why I did not see it. Thank you for the response. Did you get a chance to look at the issue I referenced? Any basic FlowLogs resource creation panics and points to this line in the code

Taking a look now!

Hi @mattzech - sorry for the delays. the since the sdk files are generated we  needed a hook to be used here. It happened that @nnbu just fixed that in #157  - so we can close this one.
Thank you both!!


Replace port type hint on the build method

You can verify this by running:
from pydantic.networks import MongoDsn

url = MongoDsn.build(
    scheme="mongodb",
    username="user",
    password="password",
    host="localhost",
    port="27017",  # use int and see 👀
    path="/test",
)

This is nonsense.


Pact Contract

this introduces support of Pact files as input contracts. In other words instead of using the Groovy DSL you can define your contracts with Pact files.
fixes #96

Current coverage is 58.73% (diff: 65.55%)

Merging #188 into master will increase coverage by 0.14%

@@             master       #188   diff @@
==========================================
  Files           198        199     +1
  Lines          5660       5845   +185
  Methods           0          0
  Messages          0          0
  Branches        955       1013    +58
==========================================
+ Hits           3316       3433   +117
- Misses         1962       1988    +26
- Partials        382        424    +42

Powered by Codecov. Last update 7ed97e1...efc7f45


[0.7] Reworking uniforms

I'm still struggling to find a good design, so this PR will likely not be merged for some time.
The "correct ways" are all impossible because of limitations in Rust.

Now rebased over #1070


Modern networks

Table of Contents
Description

Created UDP.Builder(), TCP.Builder()
Added examples

Contributing has more information and tips for a great pull request.

@zeobot set lc to pass


Script with a Source Generator doesn't compile

A script with a Source Generator fails to compile when run although intelli-sense shows that a Source Generator has run (at least during edit time) as correct xml-doc was made for the partial method.
Ext.AbcOrDefGeneratedRegex().IsMatch("abc").Dump();

public static partial class Ext
{
    [GeneratedRegex("abc|def", RegexOptions.IgnoreCase, "en-US")]
    public static partial Regex AbcOrDefGeneratedRegex();
}

@mdonatas thank you for the feedback. I haven't messed with this scenario within NetPad yet, but this is a great use case that I'd like to support. I'll check it out.

Update: I tried to create a fix for this for the upcoming release but as far as I can tell .NET is missing a couple APIs to make this possible via the implementation currently used by NetPad. I'll give it another push after the next update.

Thanks for the update and for the effort. I take it NetPad uses different pipelines for intellisense and for building as intellisense can handle generators.
Best of luck with this! There are more and more generators being used in dotnet so it would be really nice to have support for them.

I take it NetPad uses different pipelines for intellisense and for building as intellisense can handle generators.

It does. The problem I'm facing currently is I need to get the code the generator would output to include it when compiling your user code but there's no straightforward way to do that programmatically with a generator defined in the BCL (and not in your user code) that I've found so far. One rather hacky way to do it is to create a temp project and have Roslyn run the generator and grab the output but I'm trying to find a more elegant solution.


[BUG]: Simulate result AI summary not working

Scenario: [Simulate](https://github.com/DARPA-ASKEM/terarium/blob/main/testing/manual/simulate.md
Step: 3 Running simulation
Link to the project: https://app.staging.terarium.ai/projects/8583471a-9f09-49c2-8501-cf239870ba58/workflow/ab7f641f-99c3-4489-8672-d834c7bde053?operator=e40abfe4-5131-4764-95ae-f0a9de682d4b

Describe the issue
The AI summary was not generated:

There seems to be no issue here,  because the summarization relies on a generic summarization endpoint in gollm-taskrunner,  probably this, among many other things went awry with the bad taskrunner deployment.
However, the summarization should take into consideration of interventions, which currently it does not.


Bug: Executing same query multiple times should not be added to query History is previously same query is executed.

Describe the bug
Executing the same query results in it being added to the query history, even though it should not be added if the previous query executed was the same.
To Reproduce
Steps to reproduce the behavior:

Execute a query.
Execute the same query again.

Expected behavior
The same query should not be added to the query history if it was the previous query is executed.
Screenshots
N.A.
Desktop (please complete the following information):

Operating System: Windows 10
Browser: Chrome, Firefox
Browser Version: 1.61.101

Additional context
N.A.

I have the Solution Ready.

Go ahead @aditya-armal

@janvi01 can you please review this PR.


feat: add intercom

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 46.294%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/entities/DeploymentTrack/utils.ts
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
293

Relevant Lines:
451

💛  - Coveralls


Represent people inside buildings better using procedural generation

When you click a building with people inside, they currently appear, but their arrangement is a bit silly (and sometimes leaks outside the building). It'd be fun to improve this, and it's pretty self-contained. Code is at https://github.com/dabreegster/abstreet/blob/3eecdeddff70bbca2fb36f6e1bb0ef634fe01cf1/game/src/info/building.rs#L182. Some ideas:

From the building outline, procedurally generate a floor plan
Move people around inside the building slowly to represent activity inside. This should be a very cheap calculation, not a full-blown simulation or anything requiring storing any state. Ideally you take as input (sim time, person ID, the building) and come up with a position.
Maybe allow mousing over and clicking people inside buildings. At least one person during user-testing was surprised you couldn't.

Select random point in rectangle would probably do the trick for the movement, and for not leaking outside?

Many buildings aren't rectangular; they're arbitrary polygons. I don't offhand know an algorithm to generate random points in a polygon besides a brute-force generate-within-rectangular-bounds and then check.
Plus picking a random point for every person over time would cause extremely jumpy movement. I think it should be smooth if you're running the sim and watching one building. Could we do this without keeping any state? Or maybe keeping state is fine as long as the building info panel is open, but it resets when you click out.
 :)


(maint) Update integration test name

The test name when running over local is ambiguous because the test is tagged
with bash.  This commit updates the test name to better reflect what is actually
being tested.

CLA signed by all contributors.

Hey @glennsarti - if you're interested, I think having these work for powershell as well would be pretty easy. For the remote one you'd just have to add a super small powershell version of the task to https://github.com/puppetlabs/bolt/tree/master/spec/fixtures/modules/remote/tasks. The local one I think would need it's own powershell-version of the test, but that's also just copy-paste and change the shell_cmd. I'm not sure if you have time to make those changes, or care about them, but I think having these run on Windows as well would be good. Otherwise I'll just merge - this is definitely an improvement!

would be pretty easy.
So far in my experience, unpicking the bash assumptions can be a little tricky so I would definitely need to be a separate PR and work item

I'm not sure if you have time to make those changes, or care about them,
I don't have time right now (I'm working on LCOW and WCOW support for bolt already_, but I always  care about test coverage on Windows platforms


Disable logs

Hello,
I've recently begun utilizing Exile for a personal project. I've observed that Exile's logs are intermingled with Elixir's logs. Would you consider adding a feature to the library that allows the user to control this behavior via a flag?

Hi @pepicrft,
Are you referring to debug logs? They should only appear in dev env.
If you want to disable those then yes, that makes sense, I will a flag.

I see, I think the output you are seeing is from the stderr. By default, stderr is linked to console.
If possible, can you share which command are you running and what logs are you seeing?
If this is about stderr, then you can enable stderr with enable_stderr (by default it is connected to console) and just skip all {:stderr, msg}  entries in the stream.
Example:
Exile.stream!(~w(ffmpeg -i pipe:0 -f mp3 pipe:1),
  input: File.stream!("music_video.mkv", [], 65_535),
  enable_stderr: true
)
|> Stream.transform(
  fn ->
    File.open!("music.mp3", [:write, :binary])
  end,
  fn elem, file ->
    case elem do
      {:stdout, data} ->
        :ok = IO.binwrite(file, data)

      {:stderr, msg} ->
        # skip
        :ok
    end

    {[], file}
  end,
  fn file ->
    :ok = File.close(file)
  end
)
|> Stream.run()

Although this works, but I agree this lot of code for simple goal. I'll create a parameter to control this, maybe something like
:stderr with values being :console (default), :ignore (this is what you want), :stream.

@pepicrft added support for disabling the stderr output.
It is currently present in dev branch, If possible, can you give it a try?
{:exile, github: "akash-akya/exile", branch: "dev"}

You have to pass stderr: :disable option while calling Exile.stream.
Example:
Exile.stream!(~w(echo hello), stderr: :disable)
|> Enum.to_list()

Hey @pepicrft the change is merged and released v0.7.0 🎉


Updated BGP peering GA

@reasuquo : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#please-close


update dependencies

Please update dependencies to the latest version supporting null safety

It's live now .. Thanks :)


Update ready.js

Just change the emoji

If you want you can make a season resolver with the Date() method so the emoji changes based on the month instead of changing it manually

If you want you can make a season resolver with the Date() method so the emoji changes based on the month instead of changing it manually

Oh ok, I’ll try and put that in there


Flag to start at specified line/column number (#224)

I've noticed you don't have flags implemented in a way that avoids opening filenames for each flag component. I did something very hacky to get around this...please let me know how I could revise this!
Otherwise it seems to work for me.

Usually a cursor position in a file is noted as file:line:col with most editors. Not sure how conflicts are handled though.

So I was trying to follow emacs/nano:
emacs +LINE:COLUMN
nano +LINE,COLUMN
Actually, now that I think about it, maybe its easier just to check the argument to see if it has a + in the first character and then use that as a LINE,COL instead of a filename.

Okay, I've made that change.
You can start cursor on any position with micro +LINE,COL file

Please note that this behavior has been changed now since I did some flag stuff. You can now use this feature with micro -startpos LINE,COL files....


Bump version for npm update

The current version at npm/bower points to 1.1.0 although it got the additional feature to pass the 'Infinity' expiration since last month (see 738ebf0). Their referenced version is therefore out of sync and throws an error if not passing a valid date or number. As its going to be confusing for others - I recommend to publish a minor upgrade with the latest changes.

This is an issue I have just bumped into, it appears the npm package is still behind.

Hey guys,
It's published finally. I can understand the confusion - I've been using master as the main development branch, so just because something is implemented in master doesn't mean that it will be in the latest release (NPM wasn't out of date/sync - a release was never made). This is def confusing though when someone comes to the repo and reads the API reference for master instead of the latest release.
I had originally intended to make more changes before publishing a release, but everything got put on the back burner. This release has been way overdue as a result. Sorry about that!
In the future, I will likely avoid this type of confusion by using a separate dev branch and having master simply reflect the latest release.
Thanks for checking out the library!


Pmag GUI tutorial: Thellier GUI creating a pmag_criteria file

Sometime during [save MagIC tables], a pmag_criteria.txt file is being created even though we are working in data model 3.  This happens whether or not criteria have been selected (if criteria have been selected, they are added to a valid 3.0 criteria file, and the pmag_criteria.txt file is created anyway).
I'd like to track this down and figure out what is meant to be happening...

yes i noticed that.  i don’t know where it is coming from yet.
lisa

On Apr 16, 2018, at 4:08 PM, Lori J<EMAIL_ADDRESS>wrote:
Sometime during [save MagIC tables], a pmag_criteria.txt file is being created even though we are working in data model 3. This happens whether or not criteria have been selected (if criteria have been selected, they are added to a valid 3.0 criteria file, and the pmag_criteria.txt file is created anyway).
I'd like to track this down and figure out what is meant to be happening...
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/337, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDTJ7CYOemelFlC_CMZ6XPItG0cuqks5tpSSEgaJpZM4TXY-5.

That happens around here:
Thellier GUI line 4427
    # make pmag_criteria.txt if it does not exist
    if not os.path.isfile(os.path.join(self.WD, "pmag_criteria.txt")):
        Fout = open(os.path.join(self.WD, "pmag_criteria.txt"), 'w')
        Fout.write("tab\tpmag_criteria\n")
        Fout.write("er_citation_names\tpmag_criteria_code\n")
        Fout.write("This study\tACCEPT\n")

In the same section, Thellier GUI also makes a magic_methods.txt file.
I will fix it so that this pmag_criteria/magic_methods files are only created if we're in data model 2, but should a criteria file be created by default in 3.0?

i think we should only have a criteria file if there are any criteria set.  no?
lisa

On Apr 17, 2018, at 3:27 PM, Lori J<EMAIL_ADDRESS>wrote:
That happens around here:
Thellier GUI line 4427
# make pmag_criteria.txt if it does not exist
if not os.path.isfile(os.path.join(self.WD, "pmag_criteria.txt")):
    Fout = open(os.path.join(self.WD, "pmag_criteria.txt"), 'w')
    Fout.write("tab\tpmag_criteria\n")
    Fout.write("er_citation_names\tpmag_criteria_code\n")
    Fout.write("This study\tACCEPT\n")

In the same section, Thellier GUI also makes a magic_methods.txt file.
I will fix it so that this pmag_criteria/magic_methods files are only created if we're in data model 2, but should a criteria file be created by default in 3.0?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/337#issuecomment-382175816, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDVxZEKOqRu-u7q0bTokrffK4yxxrks5tpmw5gaJpZM4TXY-5.

That does make sense.  I’m not sure why the default is there for 2.5.  I will close this issue, then.


mathjax support?

Any plans to support mathjax/latex math rendering?

https://github.com/math2001/MarkdownLivePreview/issues/43