[SPARK-24900][SQL]Speed up sort when the dataset is small

What changes were proposed in this pull request?
when running the sql like 'select * from order where order_status = 4 order by order_id'. The filescan and filter will be executed twice, it may take a long time. If the final dataset is small, and the sample data covers all the data, there is no need to do so.
How was this patch tested?
Unit Test added in SQLQuerySuite.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@felixcheung
Thanks for review.
1. How small is 'small':
This optimazition works when the sampled data of the RangePartitioner covers all the data to sort.
The  size of the sampled data is :
// Cast to double to avoid overflowing ints or longs
val sampleSize = math.min(samplePointsPerPartitionHint.toDouble * partitions, 1e6)
// Assume the input partitions are roughly balanced and over-sample a little bit.
val sampleSizePerPartition = math.ceil(3.0 * sampleSize / rdd.partitions.length).toInt

val SHUFFLE_PARTITIONS = buildConf("spark.sql.shuffle.partitions")
  .doc("The default number of partitions to use when shuffling data for joins or aggregations.")
  .intConf
  .createWithDefault(200)

val RANGE_EXCHANGE_SAMPLE_SIZE_PER_PARTITION =
    buildConf("spark.sql.execution.rangeExchange.sampleSizePerPartition")
      .internal()
      .doc("Number of points to sample per partition in order to determine the range boundaries" +
          " for range partitioning, typically used in global sorting (without limit).")
      .intConf
      .createWithDefault(100)

The default value of the SHUFFLE_PARTITIONS is 200, and the default value of the RANGE_EXCHANGE_SAMPLE_SIZE_PER_PARTITION is 100, so the default size of the total sampled data is  200x100x3 = 60,000.
So when the size of the data to sort  in each partition is less than 60,000/partitions.length, this optimization works well.
In my case, 'select * from order where order_status = 9 order by order_id',  the original dataset has 10 partitions and the total size  is  20,808,930 ,  but after filtering  'order_status = 9', the total remaining size is only 18,610 (1,861 on each partition). The size of the sampled data in each partition is 60,000 / 10 = 6,000. 6,000 is larger than 1,861, so the sampled data will cover all the data to sort.
The original logic will  execute the filescan twice, the first for calculating the RangePartitioner's rangeBounds  info by sampling the data, and the second for  getting the whole data, But in this case, we have got the whole data in the first filescan, so there is no need to do the second one. This is the purpose of optimization.
2. Tests
It can be tested  by executing sql like 'select * from a order by a.b' on a dataset with a small size.
Unit Test has been added in SQLQuerySuite.
3. Benchmark
This optimazition works well when the original dataset is large, but the data to sort is small after filtering. Compared to the original logic, It will save the time on the second filescan and filter theoretically.
Benchmark has been added in SmallDataSortBenchmark.
The json dataset has 100,000,000 rows, the schema is (key, value). The key is in range (0, 100,000), each key has values (0, 1000), so the sql 'select * from src where key = $key order by value' will get 1000 rows.
before optimization:
Java HotSpot(TM) 64-Bit Server VM 1.8.0_91-b14 on Mac OS X 10.13.6
Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz

speed up sort when the dataset is small: Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
sort                                      179881 / 182216          0.6        1798.8       1.0X

after optimization:
Java HotSpot(TM) 64-Bit Server VM 1.8.0_91-b14 on Mac OS X 10.13.6
Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz

speed up sort when the dataset is small: Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
sort                                      127424 / 132435          0.8        1274.2       1.0X

the sql speeds up by 30% with this optimization.

ok to test.

Test build #94417 has started for PR 21859 at commit e4bd2e3.

Test build #94417 has finished for PR 21859 at commit e4bd2e3.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94417/
Test FAILed.

@ueshin  please test again

Test build #94424 has finished for PR 21859 at commit bf31370.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94424/
Test FAILed.

Merged build finished. Test FAILed.

Test build #94431 has started for PR 21859 at commit 58361ee.

Test build #94431 has finished for PR 21859 at commit 58361ee.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94431/
Test FAILed.

Merged build finished. Test FAILed.

This optimization is only for SQL, but other places also use RangePartitioner. What it can affect other places?

Good point. These failures may show that it affects other places.

Test build #94468 has started for PR 21859 at commit 7cc6e5a.

Test build #94468 has finished for PR 21859 at commit 7cc6e5a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94468/
Test FAILed.

@ueshin
please retest it, an unkown error occurred.

Jenkins, retest this please.

Test build #94485 has started for PR 21859 at commit 7cc6e5a.

Test build #94485 has finished for PR 21859 at commit 7cc6e5a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94485/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Test build #94496 has started for PR 21859 at commit 7cc6e5a.

Test build #94496 has finished for PR 21859 at commit 7cc6e5a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94496/
Test PASSed.

Test build #94603 has started for PR 21859 at commit 46bab16.

Test build #94603 has finished for PR 21859 at commit 46bab16.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94603/
Test FAILed.

please help retest it . @kiszk @viirya

retest this please.

Test build #94662 has started for PR 21859 at commit 46bab16.

Test build #94662 has finished for PR 21859 at commit 46bab16.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94662/
Test FAILed.

I think I need another retest . Please help. @viirya

Test build #94688 has started for PR 21859 at commit 46bab16.

Test build #94688 has finished for PR 21859 at commit 46bab16.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94688/
Test PASSed.

Test build #94931 has finished for PR 21859 at commit 46bab16.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94931/
Test PASSed.

LGTM cc @viirya @cloud-fan @gatorsmile

Test build #94939 has started for PR 21859 at commit dd6c09c.

for small queries, can we just do
val df = table.filter(...).cache()
df.sort(....)

We should carefully make trade off between the SQL engine complexity and user-benefits. How userful is this feature?

We may not know in advance how big this query is. The data at the beginning is large, but it may be very small after filtering.
I encountered this problem while using thrift server for queries. The program will query the sorted abnormal orders from the order table  through jdbc every day, the sql is 'select * from order where order_status = 4 and order_time = 20180820 order by order_id '. The number of abnormal orders is small.  It will be very inconvenient to cache the table using jdbc.

Test build #94939 has finished for PR 21859 at commit dd6c09c.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94939/
Test FAILed.

retest this please.

Test build #94947 has started for PR 21859 at commit dd6c09c.

Test build #94947 has finished for PR 21859 at commit dd6c09c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94947/
Test FAILed.

Test build #94958 has started for PR 21859 at commit ea7f317.

Test build #94958 has finished for PR 21859 at commit ea7f317.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94958/
Test PASSed.

I read the source code again.
The RangePartitioner[K, V] in ShuffleExchangeExec is an instance of RangePartitioner[InternalRow, Null]. RangePartitioner only sample K for getting the rangeBounds. So We can get the InternalRow when doing sample.
After getting the RangePartitioner, the ShuffleExchangeExec will map the InternalRow to [partitionId, InternalRow] for shuffle (the RangePartitioner generates the partitionId).
The shuffle won't use the RangePartitioner, it will use PartitionIdPassthrough instead.
In other words, the ShuffleWriter won't know the RangePartitioner's existence.
val rddWithPartitionIds: RDD[Product2[Int, InternalRow]] = newRdd.mapPartitionsInternal { iter =>
          val getPartitionKey = getPartitionKeyExtractor()
          val mutablePair = new MutablePair[Int, InternalRow]()
          iter.map { row => mutablePair.update(part.getPartition(getPartitionKey(row)), row) }
        }

 val dependency =
      new ShuffleDependency[Int, InternalRow, InternalRow](
        rddWithPartitionIds,
        new PartitionIdPassthrough(part.numPartitions),
        serializer)

private class PartitionIdPassthrough(override val numPartitions: Int) extends Partitioner {
  override def getPartition(key: Any): Int = key.asInstanceOf[Int]
}

The optimization will parallelize the cached InternalRow to the newRdd instead of getting it again.
But in other places, like rdd's sortByKey
def sortByKey(ascending: Boolean = true, numPartitions: Int = self.partitions.length)
      : RDD[(K, V)] = self.withScope
  {
    val part = new RangePartitioner(numPartitions, self, ascending)
    new ShuffledRDD[K, V, V](self, part)
      .setKeyOrdering(if (ascending) ordering else ordering.reverse)
  }
// getDependencies function in ShuffledRDD
override def getDependencies: Seq[Dependency[_]] = {
    val serializer = userSpecifiedSerializer.getOrElse {
      val serializerManager = SparkEnv.get.serializerManager
      if (mapSideCombine) {
        serializerManager.getSerializer(implicitly[ClassTag[K]], implicitly[ClassTag[C]])
      } else {
        serializerManager.getSerializer(implicitly[ClassTag[K]], implicitly[ClassTag[V]])
      }
    }
    List(new ShuffleDependency(prev, part, serializer, keyOrdering, aggregator, mapSideCombine))
  }

The rdd is [K, V], and the shuffle uses RangePartitioner directly.  But we can only get K when doing sample. so we can't restore the rdd using the cache.
They work in two different ways.
So the optimization only works in Spark Sql's ShuffleExchangeExec by now.
'The ShuffleWriter should treat RangePartitioner specially and consume the sampled data in RangePartitioner instead of the input iterator.' This idea is good, maybe we can cache both the K and V when doing sample. I will have a try.

Test build #94990 has started for PR 21859 at commit 6f52f1f.

'The ShuffleWriter should treat RangePartitioner specially and consume the sampled data in RangePartitioner instead of the input iterator.' This idea is good, maybe we can cache both the K and V when doing sample.
I will have a try on this idea.

If this optimization is done more generally, will the implicitly cached data cause memory pressure on driver, as seems we don't have way to release them?

Test build #94990 has finished for PR 21859 at commit 6f52f1f.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94990/
Test FAILed.

retest this please

Test build #95048 has started for PR 21859 at commit 6f52f1f.

Test build #95048 has finished for PR 21859 at commit 6f52f1f.

This patch fails from timeout after a configured wait of `400m`.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95048/
Test FAILed.

retest this please.

Test build #95070 has started for PR 21859 at commit 6f52f1f.

Test build #95070 has finished for PR 21859 at commit 6f52f1f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95070/
Test FAILed.

retest this please

Test build #95200 has started for PR 21859 at commit 6f52f1f.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Thank you for closing this long-running PR, @sddyljsx . As you saw here, many committers were interested in your contributions like the others. See you later in another your PR someday.


[Mapping issue] Moving blood pressure tests to measurement

Describe the problem
The following are some of the most common unmapped codes in the observation table

"1-Examination Findings-Blood pressure-Event Time"
"1-Examination Findings-Blood pressure-Laterality"
"1-Examination Findings-Blood pressure-Posture"
Would it be possible to remap them to the measument table, particularly if they have a value (I see some but not all have a value as number)?

Affected databases
cdm_gold_202301
Tag for awareness

@edward-burn It is very difficult.


Unable to detect face

Hi, guys currently I'm facing some problem on the Firebase was not able to detect the faces.
By using the original project here I able to trigger face detection with the live camera.
But I tried to create my own project it doesn't detect the face.
For Image picker, I able to get the result from my project but I'm not sure why only the Live camera face detection having this problem.
Would like to know is there any additional configuration I will need to set?
I had generated the google-services.json
This is the build.gradle file
`buildscript {
ext.kotlin_version = '1.3.20'
repositories {
google()
jcenter()
}
dependencies {
    classpath 'com.google.gms:google-services:4.2.0'
    classpath 'com.android.tools.build:gradle:3.3.2'
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}

}
allprojects {
repositories {
google()
jcenter()
}
}
`
This is the app/build.gradle
dependencies {     implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"     testImplementation 'junit:junit:4.12'     androidTestImplementation 'com.android.support.test:runner:1.0.2'     androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'     implementation 'com.google.firebase:firebase-core:16.0.8' } apply plugin: 'com.google.gms.google-services'
This is the pubspec.yaml
`
name: rf_fd_v4
description: A new Flutter application.
version: 1.0.0+1
environment:
sdk: ">=2.1.0 <3.0.0"
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^0.1.2
firebase_ml_vision: ^0.6.0+2
image_picker:
camera: ^0.5.7+4
dev_dependencies:
flutter_test:
sdk: flutter
flutter:
uses-material-design: true
`

I have exactly the same problem, any updates in this issue?

I have exactly the same problem, any updates in this issue?


Transition package from CJS to ESM

As part of the v8 effort we are currently migrating over to ESM as many packages will stop support CJS in the future. This means we have to transition all plugins to ESM as well, ideally with continuous support for CJS.

Implemented with https://github.com/webdriverio-community/wdio-vscode-service/commit/551b368847d1ab99dd3f98ae3068cb5603bf67d2


CUDA out of memory

Hi there, well done with this excellent work.
I was trying to run the demo of the model and I encountered this problem:
Successfully load checkpoint from experiment/simple3dmesh_infer/baseline_mix/final.pth.tar.
===> Start inferencing...
0%|                                                     | 0/8 [00:03<?, ?it/s]
Traceback (most recent call last):
File "main/inference.py", line 355, in
main(args)
File "main/inference.py", line 348, in main
inferencer.infer(epoch=0)
File "main/inference.py", line 124, in infer
_, _, _, _, pred_mesh, _, pred_root_xy_img = self.model(imgs, inv_trans, intrinsic_param, pose_root, depth_factor, flip_item=None, flip_mask=None)
File "/home/innova/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/innova/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/innova/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/innova/environment/VirtualMarker/virtualmarker/models/simple3dmesh.py", line 62, in forward
pred_xyz_jts, confidence, pred_uvd_jts_flat, pred_root_xy_img = self.simple3dpose(x, trans_inv, intrinsic_param, joint_root, depth_factor, flip_item, flip_output, flip_mask)       # (B, J+K, 3), (B, J+K)
File "/home/innova/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/innova/environment/VirtualMarker/virtualmarker/models/simple3dpose.py", line 202, in forward
out = norm_heatmap(self.norm_type, out)
File "/home/innova/environment/VirtualMarker/virtualmarker/models/simple3dpose.py", line 20, in norm_heatmap
heatmap = F.softmax(heatmap, 2)
File "/home/innova/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/functional.py", line 1512, in softmax
ret = input.softmax(dim)
RuntimeError: CUDA out of memory. Tried to allocate 2.53 GiB (GPU 0; 7.92 GiB total capacity; 5.50 GiB already allocated; 1.40 GiB free; 5.65 GiB reserved in total by PyTorch)
Would it be possible if you could assist me with this issue or give any advice or recommendations.
My GPU is a NVIDIA Corporation GP104GL [Quadro P4000] 8 GB
Many thanks

Edit: I reduced batch size from default=32 to default =16 and it is now working.
Edited from Interence.py file
line 49:     parser.add_argument('--batch_size', type=int, default=16, help='batch size for detection and motion capture')

Does a reduced batch size affect the model's performance? i have run the entire model and the meshes fail to be allocated to the videos.


Will there be code released on how to generate the dataset based on separate depth, seg, imname information instead of dset.h5?

I saw in some previous comments that the demo uses dset.h5, but for adding new images or creating the larger dataset, we should be reading in depth.h5 and seg.h5 separately. Since this seems to be a common need, is it possible to give more details about how to change gen.py?
Thanks for the quick responses!

No plans for this. This should be fairly straightforward.


"withStyle" vs "injectSheet"

I like to write
import withStyle from 'react-jss'

@withStyle(styles)
class Foo extends React.Component {...}

with style. 😄
I thought I'd share it, because it sounds better. Maybe consider using that name in the docs? (the default export can be named anything when imported).
Also, various libraries use "withFoo" to describe a class with a "foo" mixin or behavior. For example, SkateJS mixins are withUpdate, withChildren, etc.
The with prefix is catching on nowadays for these types of things.

I like withStyles too ... yes it is styles not style, because of the vocabulary in CSSOM. And yet I don't know if there is much value in renaming it, since it would create wording issues in projects which already use injectSheet and some co-workers might start using both ways.
Liking something is usually not good enough reason to introduce 2 syntaxes for the same thing.

Btw. injectSomething is also a popular syntax for HOCs.

useSomething is also a  popular one.


fix: Update Webhooks mapping

Updates the Elasticsearch mapping for the iex-webhooks index, and excluded some unneeded fields from being included.  This change reduces the number of fields created, because dynamic mapping creates unnecessary keyword mappings for text types.

:tada: This PR is included in version 3.5.6 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


display all error messages via console.error

Currently most error messages are suppressed in case --json flag is set. It should be updated so that all error messages will be displayed in the plain text at all time via console.error (stdout is for --json output if the flag is set).
We also don't need to display error messages in json when --json is specified. If there is an error, it should simply exit 1 with no stdout content.

Adding to this thread.. Right now, when something fails, I can't tell what went wrong or how to fix it.
Like...
hayashis@carbonate(h1):/N/dc2/scratch/hayashis/bold5000/BOLD5000_Structural/CSI1_Structural/DSI_MB3-253d_AP_CSI1 $ bl dataset upload -p 5a74ccd66ed91402ce400cc6 -d neuro/dwi -s test --dwi 20171117_114153DSIMB3253dAPSTS3S1s029a001.nii.gz --bvecs 20171117_114153DSIMB3253dAPSTS3S1s029a001.bvec --bvals 20171117_114153DSIMB3253dAPSTS3S1s029a001.bval
Waiting for upload task to be ready...
SERVICE: soichih/sca-service-noop fe
STATUS: Successfully finished
(7 seconds ago)
Dataset successfully uploaded
Validating data... (brain-life/validator-neuro-dwi)
SERVICE: brain-life/validator-neuro-dwi
STATUS: failed
Error: finished with exit code 1

We should improve the way error is handled to give user more information about what went wrong, how to fix it.


Removing links to tag pages from the blog list

For either of the below two snippets in the blog posts list page, the links to tag pages appear as blog posts. Ideally, links to tag pages shouldn't appear in the blog list page as blog posts because they aren't blog posts.
Code from Pico-Tags doc,
{% for page in pages if page.title %}

{{ page.title }}

{{ page.description }}

{% endfor %}

Code that I use in my blog list page,

        {% for page in pages|sort_by("time")|reverse %}
        {% if page.id starts with "blog/" and not page.hidden %}

{{ page.title }}

{{ page.meta.tags }}

        {% endif %}
        {% endfor %}

How do I remove them from the blog list page?

Based on what you wrote in your other issue, you could probably remove them by changing the filter from:
{% if page.id starts with "blog/" and not page.hidden %}

To:
{% if page.id starts with "blog/" and not page.id starts with "blog/tags/" and not page.hidden %}

Let me know if that works!

I had tried a variation of this earlier, but it never worked. I now tried the exact same code and it's a miss.
In both cases, I got a blank page. I mean a page with header and footer, but the content is blank. No posts matching that condition.

I was able to replicate the issue. It seems the problem is the order of execution in Twig's conditional syntax. Try this:
{% if (page.id starts with "blog/") and not (page.id starts with "blog/tags/") and not page.hidden %}

That worked for me.

This time, the blogs list, but it doesn't eliminate the tag pages. The result is pretty much the same as it was with the original code posted in the OP.
Would it help if you see the code of the entire page?

Yeah, that might help. What I posted above works fine on a test site I have locally, so I'm not sure where we're differing at the moment.

Sure, I shall post it the .twig page. If that's not sufficient, I can share the whole site as well.
--
Justin Mathew
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, December 18, 2019 1:36 AM, Pontus Horn<EMAIL_ADDRESS>wrote:

Yeah, that might help. What I posted above works fine on a test site I have locally, so I'm not sure where we're differing at the moment.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Extremely sorry. I've been trying that code snippet on the tag theme rather than the blog theme. I have tried your code now on the blog list theme and it works. The tag pages aren't shown. Thanks a lot for your help by the way, for both issues.
So my next task is to create a sidebar that displays all the tags. I am sure the same filtering can be used: to only display pages with starting with /blog/tags/ in that section.

No worries 🙂

As mentioned earlier, I am now trying to display the tag cloud on the side bar in both the blog list page and the tag pages. My content directory is as follows:
content/blog/tags/ - where the all tag pages are present which follows tags.twig theme
content/blog/ - where the blog posts are present
content/blog.md - where the blog posts are listed which follows blog.twig theme
blog.twig has the sidebar as follows,

    {% for page in pages|sort_by("time")|reverse %}
    {% if (page.id starts with "blog/tags") and not page.hidden %}
        {{ page.title }}
    {% endif %}
    {% endfor %}

The tags are displayed well. But when I try to do the same in the tags.twig theme,

    {% for page in pages|sort_by("time")|reverse %}
    {% if (page.id starts with "blog/tags/") and not page.hidden %}
        {{ page.title }}
    {% endif %}
    {% endfor %}

I do not see the tag cloud although the rest of the page is displayed correctly. I tied the following too with no success,
{% if (page.id starts with "{{ base_url }}/blog/tags/") and not page.hidden %}
{% if (page.id starts with "soch/tags/{{ page.meta.tags|lower }}") and not page.hidden %}
{% if (page.id starts with "{{ base_url }}/soch/tags/{{ page.meta.tags|lower }}") and not page.hidden %}
Indeed, the tag pages are in the blog/tags/ folder. I am not sure what is the reason for the failure here.

The issue here is that the tags pages are trying to show two different page listings using the pages array. The Filter header in the tags pages makes the plugin filter the pages array so that it only contains pages with that tag. That means we can't use the pages array as a list of all pages to e.g. display a tag cloud. That's a drawback of the plugin as it's currently built and something that could be improved upon.

Ah, I see. I tried removing the Filter from the tag pages and noticed that the tag cloud is show and for obvious reasons, the posts lists are not filtered, but all.
As you said, this is certainly something that can be improved upon later. Thanks anyway.


Remove variant from omitted text field properties for select element

Current typing causes type errors when attempting to customize the variant on the Select element.

@destos sorry I overlooked your PR and released it now in the 2.0.5 version


[WIP] No more CodeMirror

:boom: :fireworks: :boom: :tada: :boom:
This PR needs SIGNIFICANT cross browser and device testing
Please help test this PR!!
There are two easy ways to do this - the first is to set yourself up so that you're able to quickly and easily test any Ghost PR with these instructions.

Live Demo
Alternatively, you can visit http://editordemo.ghost.io and login with the following credentials:
email<EMAIL_ADDRESS>pass: test1234
Hopefully I don't have to ask that people keep the demo content clean and respectful :) I will login and clear it out on a regular basis.

This PR contains a complete reimplementation of the existing functionality we had with codemirror with a few exceptions:

the new, simpler image upload code no longer uses markers to map between the left and right panes, and instead does a quick lookup, which should work equally well on all devices

it may be necessary to change the image upload code to make sure that the event which re-enables the textarea is fired after the processing is complete, rather than after the image upload is complete (this is a maybe TODO)

now that we don't have awkward markers to keep track of, the image upload regex can be updated in both the ghostimagepreview.js and image-manager.js to be a much more powerful regex which properly supports image titles (this is a TODO)

the shortcodes have been reimplemented to be slightly smarter about where the cursor goes after, and whether or not they require a newline (more to be done here to make the list shortcut better)

the textarea does not capture TAB. This can be added, but I'm not 100% whether it's a blessing or a curse - so lets do some testing without before adding it (this is a maybe TODO)

[x] Finish up link and image shortcuts

[x] Get the editor scrolling as you type

[ ] Improve the image regex

[ ] Maybe add TAB handling

[ ] Maybe change image upload events

[x] Fix existing tests

[ ] Add new tests

closes #4368, fixes #1240 (spellcheck) and more

Drop CodeMirror in favour of a plain text area
Use rangyinputs to handle selections cross-browser
Create an API for interacting with the textarea
Replace marker manager with a much simpler image manager
Reimplement shortcuts, including some bug fixes
Rename components include editor-route-base

Windows 8.1
Chrome 39 - Flawless
IE 9 fails but 10+, works fine with spellcheck, image uploading works but is buggy. If you drag in a file that isn't a supported picture then drag in a valid picture it didn't work for me, and just shows the loading cat over the background forever (trying to upload with a 32kb jpg for IE11)

Ghost is only expected to work in the current version, and previous version of major browsers. Therefore IE9 failing is fine ;)
Sorting out the lack of indent is definitely on my radar - not sure what the right solution for it is yet, but I think we probably need something.

I have rebased this PR against latest master. I've updated the rangyinputs library now that it has been released (this fixes undo/redo on everything except IE) and I've also added spellcheck="true" to the editor.
All of these updates have been pushed to the Live demo @ http://editordemo.ghost.io (email<EMAIL_ADDRESS>pass: test1234).
There are definitely other improvements to be had: finding a solution for tabs, cleaning up some of the upload events and other things, but the base functionality is here and I think, providing we're happy this is working nicely cross-browser, it would be preferable to get this merged into master ahead of the next release and make any further adjustments as separate PRs.
@PaulAdamDavis rebasing had a lot of conflicts in CSS - it might be worth you going through and checking that I didn't screw it up (I probably did).
Pending getting the tests passing, a review of the CSS by @PaulAdamDavis and further cross-browser testing of the updated PR, I'm going to take the [WIP] tag off of this :)

I've looked over the CSS, tested in Chrome, Safari & Firefox. All the styles are as expected. :+1:
Quick note for those testing in Firefox: It doesn't properly handle padding in textareas. What the Editor does now is as good as it gets. See this bug report for more.

This fixes the spell check issues right? Is there an ETA on when this will appear in the stable branch?

I have rebased this against the new ember-cli client structure, and made some improvements. Re-raised as #5031, closing this PR in favour of that one.


hasTimestamps Doc fix

Doc lines
hasTimestamps Tests
Sets the current date/time on the timestamps columns `created_at` and
 * `updated_at` for a given method.  'insert' method will only update
 * `updated_at`.  To override the default column names, assign an array
 * to {@link Model#hasTimestamps hasTimestamps}.  The first element will
 * be the created column name and the second will be the updated
 * column name.

Should the phrase:
'insert' method will only update `updated_at.`

be changed to:
'update' method will only update `updated_at`?

Unless I am completely misunderstanding, the insert function sets both timestamps, and the update function only changes updated_at.

Nice spot. Will you provide a PR?

Yea, Ill throw one up later today.

Cheers. :)


Explain example

I don't understand the example in the readme after trying for an hour or so
Using an example from the JSON schema docs:
http://json-schema.org/example1.html
        const env = new Djv({ version: 'draft-06' })
        const jsonSchema = {
          '$schema': 'http://json-schema.org/draft-06/schema#',
          'title': 'Product',
          'description': 'A product from Acme\'s catalog',
          'type': 'object',
          'properties': {
            'id': {
              'description': 'The unique identifier for a product',
              'type': 'integer'
            },
            'name': {
              'description': 'Name of the product',
              'type': 'string'
            },
            'price': {
              'type': 'number',
              'exclusiveMinimum': 0
            }
          },
          'required': ['id', 'name', 'price']
        }

        const product = {
          'id': 'asdf', // Should be number
          'name': 'A green door',
          'price': 12.50
        }

        // Use `addSchema` to add json schema
        env.addSchema('test', jsonSchema)

        console.log(env.validate('test', product))

Outputs: type: data['id']
This error message makes no sense to me. What is it telling me? How do I parse this so I can show an error message?
It would be better to say "property id is invalid" or whatever. Or return an object with the invalid fields.
{'errors': {
'id': { 'message': 'Field is invalid' }
}}

Looking closer I see that I maybe should use the errorhandler, but I have a hard time figuring out how.
The example gives me console error:
SyntaxError: Unexpected identifier
For this line:
data: '${this.data[this.data.length - 1]}'

Thanks @Zyles for the comment. It looks like a bug, I will take a look this week.

@Zyles I've added your example to the project tests list. I've updated to main documentation for setErrorHandler as well (it was throwing an error if just copy it)
Here is a simple handler sample. However it is quite flexible, still the custom handling should be updated, and that work is already planned for the future minor release.

Here is a PR - https://github.com/korzio/djv/pull/59


CSG - Visual Bug

Godot version:
3.2.2 stable
OS/device including version:
Windows 10, 64 Bit, GLES3
Issue description:
Visual Bugs
Steps to reproduce:
See Video. CSGCombiner and 2 CSGMesh's
Minimal reproduction project:
https://www.facebook.com/daniel.suter.96/videos/10223256216141733/

Please provide a minimal reproduction project.

CSG has strange behavior

and
make strange geometry

I attach example
it have interesting Node - On :

Off :

Example:
Test.zip

#39956 fixes issues found in the provided test project. If there are other issues, please open a new issue and provide a minimal reproduction project.


support for recurrent gemma

https://huggingface.co/google/recurrentgemma-2b-it
Support for recurrent gemma

And 9b please https://huggingface.co/google/recurrentgemma-9b-it

has anyone tried loading it manually? how did it perform?

Any updates?


docs: fix typo in release notes for #653

Description
That typo was already fixed for the Boost website in commit https://github.com/boostorg/website/commit/be797166146823e39a4edc895bf65aaf419cecc2, but the change has not made it back to GIL - yet.
References
Function that was moved in #653 is named construct_matched, not construct_method.
Tasklist

[x] Review and approve

I will also merge it to master


Prompt Gym

It could be useful to create a "prompt gym" where prompts + models + parameters can be tuned and compared against our best-performing runs. We'll need some good labeled data and the ability to measure LLM output quality.

DSPy could be interesting for that


[TextInput] Add onLayout to TextInput, forwarding it to the native views

TextInput does not automatically forward all props using the spread operator so we need to explicitly forward the onLayout prop.
Test Plan: Mount a TextInput component with an onLayout prop and see that the callback handler is invoked with the TextInput's frame.

@facebook-github-bot import


core function to append to a file

It would be nice to have a core function to append to an existing file.

:tada: yeah, agreed :) It would be fairly easy to have this as a core api method, something like File.append.


AutoPyTorch on Flyte

Hello all,

 ✋
I really loved your amazing book and the work you’ve done here. I work on the “Flyte” project at Lyft and I’ve been playing around with adapting AutoPyTorch to work on Flyte.
I’m very passionate about open-source ML workflows (see here to understand what i'm doing).
I sort-of hacked the AutoPyTorch codebase in order to prove that it can work on Flyte in a distributed way. Now that I’ve done that, I’m interested in doing it the right way and I’d love to chat with someone to discuss.
Flyte handles a lot of the overhead of distributed systems (ensuring data gets from one task to the next), so I modified AutoPytorch to remove all the network calls and master/worker logic.
The new logic no longer needs a master/worker setup. Instead it runs a series of ephemeral containers, and Flyte handles moving the data from one task to the next.
The logic looks something like this (where each "-" step is a separate container):
	While True:

		- get hyperparameters (store the configs to disk)

		- run containers (one per config). fit model (using the config). Store the result and ensemble predictions to disk.

	- build an ensemble, which can be used for prediction.

In Flyte, you also get lots of visibility into where failures happen in a nice UI. It’s pretty cool to see it all working. I can tell exactly which set of HPs failed (if any).
Unfortunately, the master/worker logic is pretty baked into AutoPyTorch (and HpBandSter) right now. I had to do some very hacky things to get the POC working. I’m hoping I can chat with someone about how I can do this more properly.

Dear Jonathan,
Thank you very much for your interest in AutoPyTorch.
I think your ultimate goal would be to work on HpBandSter and not directly on AutoPyTorch.
However, please note that we don't maintain HpBandSter anymore; but our collaboration partner Bosch will release a new version of HpBandSter (in a new repo) within the next months. I think that you should talk with them then.
(Of course, we will also have a major update on AutoPyTorch then.)
Best,
Marius

Thanks @mlindauer
I'd love to sync with you on the upcoming changes, if you're willing to share details. I'm hoping to contribute to this project. I'd also love to sync with Bosch if they have the time and are looking for adopters :)

Sorry, I'm not involved in the new HpBandSter.
(My focus is more on SMAC3 (among other things))

@mlindauer Thanks, SMAC seems very interesting, I'm going to look further at that. I'm actually interested in the upcoming changes to autoPyTorch. I exploring making a large time investment in this project, and would love to know more about the direction.
I'd love to give you demo so that you can see it running on Flyte. The demo looks promising so far.


cQube - s3 file is processing is not working for diksha tpd enrolment

cQube s3 Installed in demo3 machine

Emitted the diksha tpd enrolment file to static data processor while processing it showing error please find the below error message attached below for the reference

Issue: Machine is configured with s3 buckets , here azur related configuration error is showing

Re-tested working as expected


Mask protobuf compilation warnings

Skipping compilation warnings from protobuf in clang 17

Is it throwing warnings or is it throwing errors through -Werror? The latter shouldn't happen due to it being marked as SYSTEM in the declare and we are not setting -Werror until after dependencies have resolved (at least that should be the case).

Is it throwing warnings or is it throwing errors through -Werror? The latter shouldn't happen due to it being marked as SYSTEM in the declare and we are not setting -Werror until after dependencies have resolved (at least that should be the case).

I think they are warning being print to the output. Nothing broken, but it does pollute the output whenever you need to recompile it:
[505/2105] Building CXX object _deps/protobuf-build/CMakeFiles/libprotobuf.dir/src/google/protobuf/generated_message_tctable_lite.cc.o
In file included from /home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_lite.cc:36:
/home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_impl.h:256:1: warning: function declared 'noreturn' should not return [-Winvalid-nore
turn]
  256 | }
      | ^
/home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_lite.cc:57:15: note: in instantiation of function template specialization 'google::pr
otobuf::internal::AlignFail<4UL>' requested here
   57 | template void AlignFail<4>(uintptr_t);
      |               ^
In file included from /home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_lite.cc:36:
/home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_impl.h:256:1: warning: function declared 'noreturn' should not return [-Winvalid-nore
turn]
  256 | }
      | ^
/home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_lite.cc:58:15: note: in instantiation of function template specialization 'google::pr
otobuf::internal::AlignFail<8UL>' requested here
   58 | template void AlignFail<8>(uintptr_t);
      |               ^
In file included from /home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_lite.cc:36:
/home/pedroerp/Github/velox/_build/debug/_deps/protobuf-src/src/google/protobuf/generated_message_tctable_impl.h:256:1: warning: function declared 'noreturn' should not return [-Winvalid-nore
turn]
  256 | }
      | ^


RFC: Parallel downloads.

I've been playing with this for a while now, but I'm still not sure about how to best handle the output, so I'd like to hear some feedback on the output “window”.
I have currently implemented a “window” which shows the URL and the download percentage, along with a progress bar (which is “behind” the URL using inverted colors).

So please, @Homebrew/maintainers, pull this and try it using brew download.
This is a very rough draft and far from being finished, so this should not be a code review but rather a discussion about the UI/UX.

Does this have anything that will prevent parallel downloads from the same host? That'd be a blocker from my perspective.
FYI for an external command the best way to get people to test this and provide feedback would be an external command in a tap. For this to be included here we'd probably want it to be hooked into all the DownloadStrategy stuff (which may provide another opportunity to get Cask sharing more of our code, there).

For this to be included here …

I have to clarify. This command shouldn't be included here, it only exists to demonstrate the behaviour of parallel downloads, which e.g. brew fetch and brew install would make use of when this is “hooked into all the DownloadStrategy stuff”.

Does this have anything that will prevent parallel downloads from the same host?

I'm not sure what this means exactly.

Downloading multiple files at once from the same server doesn't buy us anything in terms of performance, it just means longer waits for any one thing to download.

Downloading multiple files at once from the same server doesn't buy us anything in terms of performance

I tried with 8 parallel downloads from bintray.com vs. sequential downloads. Still 40 % faster. Then again, no one has commented yet on the behaviour. Do we want to download everything first? If so, downloading from the same host is not an issue. Or do we want to start installation as soon as one of the downloads finishes?

I tried with 8 parallel downloads from bintray.com vs. sequential downloads. Still 40-50 % faster.

Regardless of the speed it's kinda inconsiderate to the host to do so. In Bintray's case it's a CDN so I doubt they care but many hosts would.

Then again, no one has commented yet on the behaviour.

I haven't tested it as I don't understand how to test it and don't want to brew pull this currently (but an external tap would be easier).

Do we want to download everything first?

I'm not sure. If I pass e.g. brew install foo bar foo2 bar2 foo3 bar3 then the advantage is once the downloads are done they won't fail if my internet drops once connections begin. On the flip side, it's longer until e.g. foo alone will be installed. I'm probably leaning a bit towards "yes we should download everything first" partly for the not needing to download during and partly for the UI.

Or do we want to start installation as soon as one of the downloads finishes?

I think this would be technically preferable. No point letting the CPU idle while we're waiting on the network, and vice versa. Not sure how the UI would work, though.

Not sure how the UI would work, though.

Yep, that's the open question.

No point letting the CPU idle while we're waiting on the network, and vice versa.

There is a point, which is that the make-it-do-everything-in-parallel approach I think will be more confusing and increase support our burden. Also, waiting for everything to download before starting any installs seems really annoying in exchange for a minor benefit to those who are using brew with intermittent network access. So personally I prefer the current behavior to anything be proposed here.

I think I'm 👍 (and based on Slack conversations with @ilovezfs he also seems to be) on implementing this now as a brew fetch --parallel option (possibly also enabled by default with an environment variable). If that ends up being a solution to a problem and not buggy we could look at promoting it to the brew fetch default. Only after we've evaluated all that would it be something we'd think about for brew install's flow, I think.
As mentioned before: personally I don't find it useful to test a command like this without looking at the code in a way that could be integrated so I'd suggest this PR moves instead towards the brew fetch --parallel approach.

Closing this as it can still be brew pulld, discussion has stopped here in favour of a future PR adding the functionality to brew fetch.


Add a validation endpoint filter for minimal APIs

Make it simple to do validation on multiple endpoints via an endpoint filter (new in ASP.NET Core 7).

Needs more tests.


The current utils/selector doesn't work with 1.13.

The current utils/selector can't parse nbt= arguments.

Will be fixed when Minecraft 1.14 comes out.

I'm working on it...


diacrisis, geoscientists

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[d,g]

/trunk merge


Ability to set a file-like object (instead of a file path) for stdin and stderr

I want to use wasmtime-py to run untrusted Python code safely inside a WASM sandbox. Here's the recipe I have so far, thanks to help from @pims https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox
Part of that recipe is this:
config = WasiConfig()

config.argv = ("python", "-c", code)
config.preopen_dir(".", "/")

with tempfile.TemporaryDirectory() as chroot:
    out_log = os.path.join(chroot, "out.log")
    err_log = os.path.join(chroot, "err.log")
    config.stdout_file = out_log
    config.stderr_file = err_log

    store = Store(linker.engine)

    # Limits how many instructions can be executed:
    store.add_fuel(fuel)
    store.set_wasi(config)

Then the code reads from that out.log temporary file after executing the code.
It would be much neater if it was possible to pass a file-like object here instead - a io.BytesIO or io.StringIO for example.

@pims points out that this can be done using the Go bindings like so:
body := []byte{}
in := bytes.NewReader (body)
out := new(bytes. Buffer)
stderr := new (bytes.Buffer)

config := wazero.NewModuleConfig().
    WithFS (fs) .
    withstdout (out) . withStdin(in).
    WithStderr(stderr).
    WithName (name).
    WithArgs ("python", "my app-py").
    WithRandSource(rnd)

Minor correction: using a different WASM VM, wazero, as opposed to wasmtime.

I just noticed this may be a duplicate of:

https://github.com/bytecodealliance/wasmtime-py/issues/34

Also relevant:

https://github.com/bytecodealliance/wasmtime/issues/4372
https://github.com/bytecodealliance/wasmtime-go/issues/34

Thanks for the report! I think this is the same as https://github.com/bytecodealliance/wasmtime-py/issues/34 so I'm going to close in favor of that.


Added ability to run benchmarks in podman containers

Added ability to run benchmarks in podman containers. Functional, but needs cleaning and polishing things like static paths.

looks good!
true/false/local/container :) we messed up :)
pls get ird of hardvoced apths:

/home/tester/diplomka/test-docker
CONTAINER_PATH=/home/tester/diplomka/test-docker
maybe more

pls move cosntants to variables

JDK_dockerfile
preparation-con

get rid of many unused code

the comments, and well.. you knowbetter
jdk_path/nameis unsused insome scripots

There is unhapy concept of various cd known_dir
It was ok in the previous vm scripts, where the warorse was one script.
Now it is three scripts, and the path have to  resued in them.
Idea - all logging will go to &2 and the output of  the prepare_container.sh will be only the used dir, lets say exported to some global variable, and reused by other two scripts.
Or maybe reuse that $WORKSAPCE?
Thanx!


Misleading about information of repo

I found that the about information of this repo is misleading:

Definitely, we support Erlang/OTP >= 24.
Maybe it's better to use the same description: "Building native-like Elixir apps for Windows, MacOS, Linux, iOS and Android using Phoenix LiveView!"

updated!


Fix custom scale for Home Assistant 2024.11

While playing around with the card, I've noticed that the example for custom scales in the docs did not work.
It threw this error message:

Apparently, Home Assistant at some point started making the config passed to the card immutable?
Cloning it did fix the issue.
Using JSON.parse(JSON.stringify(thing)) for cloning is a bit old-school. You'd normally use structuredClone() now.
https://caniuse.com/?search=structuredClone
I did not know what your goals are for this card so I picked the old solution that works in old browsers as well

this looks like the fix for https://github.com/kandsten/ha-heatmap-card/issues/15

Thanks; it's something that should have been fixed a long time ago. The sad truth of the matter is that my mind has been wandering elsewhere and I haven't paid quite enough attention to this card - something that's kind of needed tracking a (relatively) young project like HA.
In terms of goals and in the light of the above, I'm somewhere between "hand this over to someone who appreciates the idea and has the drive to take it on, further develop it" and keeping the lights on and adding some scales every once in a blue moon.


fix for Crystal 0.35.0

Another breaking-change in IO is that #skip, #write, #write_utf8, #write_byte, #write_bytes, and #skip_to_end return the number of bytes it skipped/written
https://crystal-lang.org/2020/06/09/crystal-0.35.0-released.html

Removing the type restrictions and letting the compiler figure it out worked for me in a project that uses this. ~I think that's all that needs to change here, but I'm not 100% sure~
edit:
I saw on travis it complained about
In src/channel.cr:231:9

 231 | def write(slice : Slice(UInt8))

           ^----

Warning: this method overrides IO#write(slice : Bytes) which has an explicit return type of Int64.

Please add an explicit return type (Int64 or a subtype of it) to this method as well.

The above warning will become an error in a future Crystal version.

In src/channel.cr:105:7

 105 | def write(slice : Slice(UInt8))

           ^----

Warning: this method overrides IO#write(slice : Bytes) which has an explicit return type of Int64.

Please add an explicit return type (Int64 or a subtype of it) to this method as well.

The above warning will become an error in a future Crystal version.

In src/sftp/file.cr:29:9

 29 | def write(slice : Slice(UInt8))

          ^----

Warning: this method overrides IO#write(slice : Bytes) which has an explicit return type of Int64.

Please add an explicit return type (Int64 or a subtype of it) to this method as well.

The above warning will become an error in a future Crystal version.

A total of 3 warnings were found.

so I redid this with making them Int64 instead of nothing.

Thanks @will much appreciated

cut new release v1.4.0


/bin/sh: mtxrun: not found (Docker on Ubuntu 20.04)

Expected Behavior
On running docker-compose up -d, resume.pdf should be generated.
Actual Behavior
No PDF is generated (.html and .tex are both created, apparently without error). output/context_resume.log shows the following:
cat output/context_resume.log
/bin/sh: mtxrun: not found

On further investigation, it seems that while pandoc is installed fine:
% docker run -it resume-make sh
/data # pandoc --version
pandoc 2.9
Compiled with pandoc-types 1.20, texmath 0.12, skylighting 0.8.3
Default user data directory: /root/.local/share/pandoc or /root/.pandoc

ConTeXt is not:
/data # context --version
sh: context: not found

Steps to reproduce the behavior
git clone https://github.com/mszep/pandoc_resume
docker-compose up -d

Versions
Pandoc and ConTeXt versions as above.
OS is Ubuntu 20.04.3 LTS.

It looks like mtxrun is not included in the pandoc/latex container image.

I ran into this as well.


Fix for Applied Energistics 2 SAG Mill Recipes

Changed to ore dictionary entries where available, as well as new item ids.

is this really for Minecraft 1.7.10?

My bad, it is for 1.10.2. I will re-submit it.

ok, thx.


(PUP-3182) Call Indirection#save with the correct arguments

Previously, puppet catalog download failed with an obscure error:
Instance name "<hostname>" does not match requested key {}

The problem occurred because Puppet::Indirector::Face called
Puppet::Indirector::Indirection#save(key, options) with the wrong set of
arguments. The save method expects to be called with:
def save(instance, key = nil, options={})

Note how the instance to save should be the first argument, key is the
second, and options is optional. But Face was passing key as the instance,
and options as the key, leading to "does not match requested key".
The code originally worked in 2.7.x, but passing options never worked
correctly for any face application. During a security event, we added
validation to the Indirection methods, ensuring the instance being
passed in matched the key that the client was authorized for[1]. So for
example, you can't be authorized to send a report, but send something
else in the body of the request. This change was made in
f877cf5d63ea4b6d3bc110af6212e5187f900ee9, and released in 3.1.1, but
broke saving face-related models.
This commit modifies Puppet::Indirector::Face to call the save method
with the correct arguments. It also updates puppet catalog download
to output where the catalog was downloaded to:
Notice: Saved catalog for <host> to /opt/puppetlabs/puppet/cache/client_yaml/catalog/<host>.yaml

[1] http://projects.puppetlabs.com/issues/19392

https://access.redhat.com/solutions/1373553

CLA signed by all contributors.


BLE-OTA-v1.4

Hi,
Is this android already support your new BLE arduino with fatfs?
Newest apk binary build is BLE-OTA-v1.4.apk

As far as I know this repo has only v1.0.
Sadly, the v1.4 is not here in git. The source code of this version would be very interesting for me because I want to implement this OTA behaviour on a different app connecting with a different microcontroller. @fbiego is it possible for you to update the source code in this repo? That would make me very happy.

fbiego, you are the best! :)

Thank you fbiego, closed now.


[Desktop] Add UI transparency

Add transparency in the locations shown below

Issue created by BlueBubbles Bot in the BlueBubbles commnity on behalf of @Randymations in #desktop-support.

Additional transparency in these sections, but less than others


Fix the datetime return formatting

This pull request resolves the issue where the date return includes time which is not really desirable. This update uses the time.Format method to return the date in a normal ISO format that is easier to read and understand.

thanks for continuing development on this and sorry I havent been as active as before. Looking at this code is like hugging an old friend now that my day to day is primarily in java :)

thanks for continuing development on this and sorry I havent been as active as before. Looking at this code is like hugging an old friend now that my day job is primarily in java :)

Big F dude sorry you have to write Java haha.


Search: Issues when getting input element ref

It's unclear what the expected way to get a ref to a Search component's <input> element is. The methods I've been able to make work are a little less than ideal for various reasons.
Use Case
I have a Search component that stays mounted permanently at the top of my page. In the onResultSelect callback I trigger route changes (using react-router), however the search input retains focus. That behaviour might be desirable in some cases, but in this case I would like to unfocus the input when this happens, which means getting a handle on the DOM element so I can call .blur() on it.
Actual Result
These are the methods I've been able to make work.

Nested shorthand props:
// In class body
inputRef = React.createRef()

// In render
<Search
  ...
  input={{
    input: { ref: this.inputRef }
  }}
/>

Problem: The <input> element loses the props that Search sets, the ones found here:
https://github.com/Semantic-Org/Semantic-UI-React/blob/dd66e920a845c199ad2cc05ddec38293a235ec20/src/modules/Search/Search.js#L531
This is because the linked code doesn't account for the possibility of having to merge the nested shorthand props, instead the object I provide just overwrites them. Of course I could manually add those props myself, but that seems hacky/brittle. Should this be considered a bug?

Undocumented property on Input component:
// In class body
inputcomponentRef = React.createRef()

handleResultSelect = (e, { result }) => {
  ...
  this.inputComponentRef.current.inputRef.current.blur()
}

// In render()
<Search
  ...
  input={{ ref: this.inputComponentRef }}
/>

Problem: Since it's undocumented, it shouldn't be relied on. See #1980.
Actually the Input component provides a (documented) .focus() method directly, presumably to make this sort of thing easier. Since that's the case, maybe its .blur() counterpart deserves to be added too? But I feel like that's a slippery slop that just skirts around the issue.

Function shorthand + Ref component:
// In class body
wrapperDivRef = React.createRef()

handleResultSelect = (e, { result }) => {
  ...
  this.wrapperDivRef.current.querySelector('input.prompt').blur()
}

// In render()
<Search
  ...
  input={(Input, inputProps) => (
    <Ref innerRef={this.wrapperDivRef}>
      <Input {...inputProps} />
    </Ref>
  )}
/>

This would be nicer if the .querySelector() call could be avoided, but the markup requires the wrapper div. Also this uses ReactDOM.findDOMNode() under the hood, which is deprecated. It also seems like this might have slight performance implications? I realize I can extract the function to stop a new one being created on every render and maintain referential consistency, but there's still the overhead of the HOC if nothing else.
Probably still the best option currently.

Expected Result
I get a reference to the <input> element in a 'clean'/user-friendly manner.
Version
0.87.3

The first solution is the best. It seems that shorthand in Search should be fixed to work properly.

@layershifter PR submitted.


Moving to azure-iot-device preview 6

Updating casing of imports for consistancy

Closing this PR without a commit as new changes with larger fixes are coming


Date (of today) automatically shown in TopNav

Value Proposition
As a ...user
I need ...the day of today
to be able to ...see the date of the day immediately and automatically.
Description

Acceptance Criteria

[ ] The date of the day has to be shown
[ ] Using a library in React to fill in the date automatically when open the app

Tasks
(for developers team)

[ ] Create feature branch
[ ] Create component
[ ] Update components
[ ] Create styled components
[ ] Write tests
[ ] Storybook

Hey Yvonne,
I really like your user story. 👍  I only have two remarks:
Description
On the right we can also see a monthly calendar view. I think this will be too small for users to see or? 🤔 Therefore, I just asked myself if it is necessary to implement this?
Acceptance criteria
"Using a library in React to fill in the date automatically when open the app" - This is not a criterion that a user will see. This is an implementation detail and should be moved to tasks.


[Feature] Arbitrary Mesh Display

What problem does this feature solve?
eCharts GL and 3D rendering is very restricted at the moment when wanting to visualise 3D data outside of scatter/plot or a very limited "polygon" (which is really just extruded 2D shape)
As web browsers and WebGL get more powerful,  there is opportunity to visualise data in new ways in 3D.
eCharts would be improved in their 3D/GL area with the ability to have arbitrary mesh support:
Use cases:

Medical application where complex 3D models of organs can be rendered
Mining and other large scale (geographically) projects needing to visualise spatial data
Arbitrary meshes as MarkPoints (a 3D human mesh,  a 3D car mesh on a road,  other 3D markers)
Markpoints and Markareas as sub-meshes ,   think of a medical application where certain surface patches have Tooltip data

What does the proposed API look like?
In the Polygon series type ( https://echarts.apache.org/en/option-gl.html#series-polygons3D.data )
You can only set XY of the data and then a constant height for everything.
It would be great to support rich 3D mesh data,  here is simple proposal of a triangle
data:[
{
vertices: [[0,0,0],  [0,1,0],  [0,0,1]],
triangles: [ [0,1,2] ] //  Indexes of the vertex buffer with a stride of 3
uv:  [[0,0],  [0,0.5],  [0.5,0]   //  UV coordinates of vertices for texture mapping
}
]
How we feed these arrays could then be arbitrary...  from a proceedural process,  or by processing a popular 3D file (STL, FBX, OBJ)
Generally, the mesh data could include

Sub-meshes to denote different materials (  data:[ {meshA},  {meshB} ]  )
Multi-stream Vertex (position, normal, UV, tangent, bitangent, RGBA color)
Tooltip support?  (based on submesh?  or different polygons ,  vertices, edges?)

Hello - was curious if there were any thoughts on this or if there were any plans for the GL component of eCharts looking forward?

sorry to bump again,  would be great to hear of any roadmap or upcoming features for GL and eCharts without needing to move this part of our application to a different library as we have built a nice framework with eCharts

shame there's no comments on this?
the GL pack of eCharts is a promising start and would be great if it were extended rather than having to jump to another package (ie. we can still use much of the eCharts Instance/boilerplate code and tooltips... etc)
is there any roadmap at all for eCharts (not just GL) ?

Bumping this as well, that would be a great feature to have!


chore(action): Update actions/core and actions/github dependencies

Motivation
Updates @actions/core to 1.10.0 and @actions/github to 5.1.1. The @actions/core update resolves deprecation warnings coming from calls to core.setOutput(), which will be removed in June 2023.
Approach
Updates dependencies
Possible Drawbacks or Risks
N/A
TODOs and Open Questions

[ ] Rebuild the action dist (I can do this if you prefer)

Cheers! This action dist update should be handled by the workflows. Please update the related package.json versions to suit and add a change file, then it should be good to go.

Sorry for not responding @jbolda ! Was busy with exams last week. Thanks for merging!


[Question] Multi-output model

Hello Dear @BrikerMan .  Thanks for creating Kashgari! I tried the example in mutli-output link (https://kashgari.bmio.net/advance-use/multi-output-model/)  but it raises an error.
Thanks.
kashgari/processors/classification_processor.py", line 35, in _build_label_dict
label_set = set(labels)
TypeError: unhashable type: 'list'

@BrikerMan you can change line 35 in classification processor to the following. (https://stackoverflow.com/questions/57763803/convert-a-list-of-lists-into-a-set-of-lists-in-python-3)
label_set = set()
for array in label_set:
label_set.add(tuple(array))

Hi, When you using the multi-output model. You need to customize your own processor, too. Just subclass the classification processor and change it.

@BrikerMan thx


TC-SWTCH-1.1 - Feature map should support either LS or MS flag as per spec.

Problem:
As per spec the implementation of the feature is mandatory for LS or MS . But feature map value shows as "0" while executing.
Actual outcome:
./chip-tool switch read feature-map 1 1
[1646208136.242043][2390:2395] CHIP:TOO:   FeatureMap: 0
App used - all clusters app
platform - Raspi
spec screenshot for ref:

spec link:
https://github.com/CHIP-Specifications/connectedhomeip-spec/blob/master/src/app_clusters/Switch.adoc#features

Duplicate of #19228


fix async allreduce algorithm

Fix exit error and check speed/loss results in CI

fixed in https://github.com/BaguaSys/bagua/pull/185


SPFarm: Add SPServerCertificatePipeBind Support for the Central Administration with SPSE

Problem description
SharePoint Subscrioption Edition added a few Parameters for the SPCentralAdministration Commands. The Parameter -Certificate  is one of them. In conjunction with the Parameter HostHeader,  UseServerNameIndication, AllowLegacyEncryption and Url it is now possible to create a Central Administration with an Certificate Binding and a URL of your choosing.
Also with the benefit, that a .Provision() on the Webapplication or a psconfig run does not revert the URL to the default or removes the Binding from IIS.
Also you could run two or more instances of the Central Administration behind a Loadbalancer
Currently this is not possible with the SPFarm Ressource.
Verbose logs
-

DSC configuration
-

Suggested solution
In order to achieve this the following Steps would be required:

Test if the Cerfificate is imported before running the New-SPCentralAdministration or Set-SPCentralAdministration Command and Import it if necessary
Expand the Parameters for the SPCentralAdministration Commands
Expand the Ressource with the Addional Parameters

I'd be happy to work on a pull request.
I'm a bit concerend that this might get in conflict with the existing SPCertificate Ressource, but also could reuse the existing code :)
Also I'm happy about feedback and if this enhanced fits in.
SharePoint version and build
-

Operating system the target node is running
-

PowerShell version and build the target node is running
-

SharePointDsc version
-

As an alternative a new Ressource SPCentralAdministration  would be possible. Which could be used after SPFarm and SPCertificate. The Ressource would just contain the Certificate  and AllowLegacyEncryption Parameter.
Also the Hostheader Parameter would be added to SPFarm.
One thing about SPFarm - the reprovisioning of the Central Admin should be done with Set-SPCentralAdministration on SPSSE.


Update Packet Generator provisioning code to support nfvbench 2.0.3, multi-chaining tests and reserved/on demand instances

Ansible tasks for - Update Ansible playbooks to match expected environment and configuration:

[x] Create and test ansible playbook for intel

[x] Configure interfaces to match configuration

(X710 4x10GbE) 1) Bond0, 2) VLAN_1, 3) VLAN_2, 4) Unused

[x] Add additional steps for using Intel NIC

Load vfio-pci kernel module at boot

[x] Update NFVbench role to use NFVbench 2.x

details TBD

[x] update the packet_generator deploy script to support provisioning existing nodes.

Terraform tasks:

[x] update deploy scripts to use different TF override files
[x] create a TF override file for the dual port that's empty
[x] create a TF override file for the quad port intel that uses reserved instances

[ ] Integrate packet generator updates from #92 as of Nov 7th, 2018
details TBD

@pmikus, if your available help with integrating the packet generator changes from #92 into the oprovisioning code  (ansible and deploy config) would be appreciated.
The Ansible code can be tested w/o the terraform or together.

@pmikus, if your available help with integrating the packet generator changes from #92 into the oprovisioning code (ansible and deploy config) would be appreciated.
The Ansible code can be tested w/o the terraform or together.

With my pleasure. Looking on it.

Update:
Peter Mikus [8 hours ago]

After testing and bug fixing I can confirm that 2.0.3 is the version that works smoothly now.

I update all instances of packet generator code.
We may want to override VLAN setting of nfvbench to match destination. But other then that, done,

@pmikus, we can add the VLAN setting override in another issue after #113 is done.


Long dropdown being rendered full width of window

In 6.1.1
Reproducing:

Create a dropdown element
Contents of dropdown needs to be higher than the height of the window
Open the dropdown
The dropdown is rendered with a width that matches the width of the window, so it appears super-wide

What I'd expect
I would expect the DD to keep it's native size, overflow past the bottom of the window, thus allowing the browser scrolling to be used. Or even better, supporting scrolling inside the DD using overflow css setting.
It looks like it's an issue in Dropdown.prototype._setPosition, but not knowing the history of the function I'm reluctant to offer a fix

I'm having the same issue. Any progress on this?

A temporary fix could be to add a max-height and overflow attribute to the .dropdown-pane.
.dropdown-pane {
  max-height: calc(100vh - 80%) // could also be a pixel or rem value
  overflow: auto; // display a scroll bar if the content in the dropdown exceeds the max-height
}

I've verified that this works in Chrome, Firefox and IE.

Having the same issue.


Improve provider coverage via code generation

What problem are you facing?
Coverage of the GCP API here is somewhat low and the overhead of adding individual managed resources is rather high and somewhat slow going. Platform engineers working within the context of a cloud provider will need to cover considerable amount of API surface area not yet found here. Org policies, projects, folders, all the network primitives, log sinks, ect. are necessary for security, platform, and network engineers to effectively serve internal engineering teams/customers. This amounts to some difficulty in recommending crossplane to GCP customers in the current provider state.
How could Crossplane help solve your problem?
Code generation of the CRDs and types would significantly move the needle on API coverage and potentially grow adoption amongst GCP users who already tend to leverage k8s more than customers of other cloud providers. Mechanisms would need to be developed to override or omit portions of the generated code base with known issues/limitations.
It appears a code generation approach has been taken in provider-aws and GCP is (in my opinion) an easier target in this regard. While not necessarily the best way to generate a client (i.e. lots of inconsistencies), pulumi-google-native uses GCP API discovery documents to generate their provider. Many learnings could be gathered from the prior art there. A large subset of protobufs in googleapis cover the GCP API surface area in a more robust and consistent way. DCL is another up-and-coming approach that the existing google-terraform providers seem to be progressively migrating toward - not as clean a fit for what I'm suggesting here but worth mention.
Caveats
It shouldn't go unsaid, there are trade-offs to codegen. Namely, a faithful representation of a provider's API isn't necessarily the most correct interface to expose in a declarative context. Related, some quality is sacrificed for raw quantity, particularly upfront. I think the terrajet generated gcp provider is a fair rebuttal to these arguments since it embraces generation based on a large body of mostly handcrafted work that serves an existing customer base well. Still, both paths (a provider generated from protos/DD and a provider generated from existing tf provider schema) feel like they have distinct value.
Controllers and clients appear much less uniform than the other parts of the codebase and arguably are the meat of the provider. Unsure if there's sufficient value in the codegen proposed here if controllers can't be effectively generated.
Finally, development/contributor onboarding complexity likely increases as codegen layers are added. This is likely a worthwhile cost on balance but worth stating. I'm sure you've seen some of these caveats play out within the AWS provider.
Note
Apologies in advance if this direction has been considered and discussed - I was unable to find such discussions in github but perhaps it's buried in slack.
Also, this project (i.e. crossplane) is 💯 the future. All the work here is much appreciated. 🙏

Thank you for bringing this up @brandonjbjelland and for sharing your well informed ideas
@muvaf wrote up a design doc about the strategy that the Crossplane project will use for achieving provider coverage in the short/med/long term, you can read all the details here: https://github.com/crossplane/crossplane/blob/master/design/design-doc-provider-strategy.md
tl;dr: Short-Medium term will be the new jet providers, e.g. https://github.com/crossplane-contrib/provider-jet-gcp.  See all the GCP resources available in https://doc.crds.dev/github.com/crossplane-contrib/provider-jet-gcp.
Long term will be continuing to partner with the cloud providers themselves and their code gen tooling/pipelines to generate native Crossplane providers that are tightly synced with the cloud provider roadmaps.

My fault @jbw976 - I very clearly missed that doc! @muvaf 's analysis looks exceptional. 👏  With as much work is on your collective plate, it's probably the right move leapfrogging DD/protobuf GCP sources in favor of DCL. Looking forward to that eventual future!
Re: provider-jet-gcp - I noticed the most recent release 0.2.0-preview was essentially what I'm after but given that it's the first and only branch/tag having that large set of resources, I had some pause. I'll keep an eye on provider-jet-gcp as it matures.
That resolves core issue here. Super appreciate your response 🙏


Soft Crash whenever attempting an autocrafting order from an Arcane Crafting Terminal #410

Reposted from https://github.com/Nividica/ThaumicEnergistics/issues/410:

Bug Desctiption: Whenever I attempt to put through an autocrafting order from an arcane crafting terminal, my game soft crashes. The prompt to select the number of items to autocraft pops up, but when I hit next, the game soft crashes before the next prompt showing the items to be crafted shows up.
Steps to reproduce the behavior:

Open arcane crafting terminal
Select an item to autocraft
select any quantity and hit next
Game then soft crashes

Expected behavior The next prompt showing the "items to be crafted" and then clicking the start button.
Screenshots
Version Information

Minecraft: 1.12.2
Forge: <IP_ADDRESS>
Applied Energistics: AE2 Unofficial Extended Life v0.56.4
Thaumcraft: 6.1.BETA26
Thaumic Energistics: 2.2.7
Modpack: The 1.12.2 Pack

Logs/Crashlogs https://mclo.gs/Y2XPEgT

There's been a few posts on this from the DJ2 discord also, and your crash log matches theirs.
I remain uncertain why this behaviour occurs. When the gui is initialised, we get the number box from the AE2 gui via reflection and hold onto it, later trying to get the craft quantity when you click next.
The code already tries to fail early if it cannot get the number box at initialisation time. I don't know why it nulls out when we click next, I have been unable to reproduce it.
However I have rewritten all the code around this area, removed the reflection and instead deferred accessing any element of the AE2 gui until as late as possible. Hopefully by this point, everything is initialised fully. It may help.

@Delfayne If you have a test build to share, I can send it to the player who reported the issue to confirm whether it still occurs.

@Delfayne If you have a test build to share, I can send it to the player who reported the issue to confirm whether it still occurs.
FixBuild.zip

Here is the fix build, containing the mod files for GalactiCraft and ThaumicEnergistics

https://github.com/Delfayne/ThaumicEnergistics/releases/tag/v2.3.0
@xJon @zeng-github01
You can find a pre-release version here, I will push out to CF at the weekend if testing goes well.

Fixed in https://github.com/Delfayne/ThaumicEnergistics/releases/tag/v2.3.0

@Delfayne Any reason 2.3.0 is not uploaded to CurseForge?

@Delfayne Any reason 2.3.0 is not uploaded to CurseForge?

Some concerns were raised when some DJ2 players imported the mod and it behaved strangely for them. It's an isolated report, but I wasn't sure if there was a bigger problem.
Have you tested 2.3.0 without issues?

@Delfayne Not yet tested, but I will report soon if I encounter any issues 👍


ModuleNotFoundError: No module named 'datasets' when running generation

Hi, when I was running 'python generate.py --ckpt_path ${model_path} --in_image examples/sample0.png --in_mask examples/mask0.png --text wizards --output_dir ${output_dir}', there was an error: 'ModuleNotFoundError: No module named 'datasets''.
I didn't see any dataset folder in this repo, how to solve this problem?
Thanks!

Close this duplicated issue.


Flutter Daemon has terminated

Issue Type: Bug
I'm getting the errors as

The Flutter Daemon has terminated
2)THe Dart Analyzer could not be started
Can you help me in fixing this problem

VS Code version: Code 1.43.0 (78a4c91400152c0f27ba4d363eb56d2835f9903a, 2020-03-09T19:47:57.235Z)
OS version: Windows_NT x64 10.0.16299

System Info

Item
Value

CPUs
AMD A6-9220 RADEON R4, 5 COMPUTE CORES 2C+3G    (2 x 2495)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: disabled_offoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
3.89GB (1.33GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (4)

Extension
Author (truncated)
Version

android-dev-ext
ade
0.8.0

dart-code
Dar
3.8.1

flutter
Dar
3.8.1

adb-interface-vscode
vin
0.14.0

Please file issues against the Flutter extension you are using, thanks!


Update example to use item reader/writer builders

This PR updates the example in the getting started guide to use item reader/writer builders.
Note that this PR is based on the upgrade-to-boot-2 branch (since builders were added in batch 4.0.0.RELEASE brought by boot 2.0.0.RELEASE).

Hi Greg,
One of the main themes of batch v4 is the addition of builders for item readers/writers and almost all Java config examples in the reference documentation use them. So I thought it would be a good idea to have the starting guide in line with the rest of the docs (now that boot 2 brings batch v4).
What do you think?
Best regards,
Mahmoud


Changing Death Voice Quotes for each Opponent

Describe your problem here.
I need help for this!
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No response

uh


Docker image of jmx_exporter

Hi there,
first thank you for implementing this solution. I try to connect prometheus and the kafka via the jmx_exporter on an openshift cluster.
When using the image for he jmx exporter compiled by the the Dockerfiles you provided, I get the following error message in the prometheus web ui:

Any idea how I can resolve this error?
Thanks!

I think I set some ports in a wrong way, now its working.


Any way to track utm parameters?

Hey, I am loving Shynet so far. Well done! However, I would like to track users who come from different sources via utm parameters. Is there any way this can be done?

This is not yet implemented, but it's something that we should probably incorporate. Adding this to the roadmap.


fix: eks 1.28 migrate kubectl provider to alekc repo

What does this PR do?
Fix issue with gavinbunney/kubectl on eks 1.28. Some resource disappear or re-created on every tf apply
Migrate to alekc/kubectl providers
close #244
Motivation
Make it work on every tf apply with eks 1.28
More

[x] Yes, I have tested the PR using my local account setup  (Provide any test evidence report under Additional Notes)
[x] Yes, I ran pre-commit run -a with this PR

For Moderators

[ ] E2E Test successfully complete before merge?

Additional Notes

Hey thanks for this PR!

Hi, weird i had miss a commit. sorry
LGTM now without tfsec
[] 🠶 pre-commit run -a
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check for merge conflicts................................................Passed
detect private key.......................................................Passed
detect aws credentials...................................................Passed
Terraform fmt............................................................Passed
Terraform docs...........................................................Passed
Terraform validate.......................................................Passed
Terraform validate with tflint...........................................Passed
Terraform validate with tfsec............................................Failed


internal/pkg/scaffold/helm/chart_test.go: add license header

Description of the change: add license header to internal/pkg/scaffold/helm/chart_test.go
Motivation for the change: needs license header.

@joelanford internal/ is not vetted. #1475 fixes that. I can break that out into a separate PR since its an unrelated change.


Set a blank excerpt for all pages

This commit sets a blank Jekyll excerpt as the default for all pages
(docs on excerpts). Without this configuration, Jekyll tries to
generate a page excerpt using the first paragraph of each page. However,
we do not use the page excerpt anywhere in this project (since we don't
have an equivalent of a "list of all blog posts" page). Setting a blank
excerpt decreases website build time.
More importantly, in a later commit I plan to wrap HTML generated from a
Jupyter notebook in a {% raw %} tag (docs). Without this tag,
complicated LaTeX will break Jekyll builds. However, when Jekyll tries
to generate page excerpts, it breaks because the excerpt contains the
{% raw %} tag but not the {% endraw %} tag.

This sounds reasonable to me... I guess this does not affect social media excerpts?

Oh good catch, I didn't think about social media excerpts. We can generate the excerpt for while converting notebooks to Markdown and leave it in the YAML front-matter for each page. I'll see if I can get that working.

Ah nice - that makes sense to me. Maybe just the first N characters or words?

Here's where the social media tags live:
https://github.com/jupyter/jupyter-book/blob/master/jupyter_book/book_template/_includes/fb_tags.html
If the excerpt isn't there, then it just defaults to the site description. To be honest that sounds fine with me too...

The fix ended up being simpler than I thought: instead of using page.excerpt to generate social media tags, we can use page.content instead (which always exists).

Ah nice - that's a clever solution. OK let's 🚢 it!


Transaction detail page should go away

The input modal of the transaction should go away after the transaction is successfully broadcasted.

The input modal of the transaction should go away after the transaction is successfully broadcasted.

@kwunyeung is there any step to reproduce this? currently all modal will be closed after transaction is broadcasted

probably a brave browser websocket issue

Please add your planning poker estimate with ZenHub @calvinkei

@kwunyeung  it turns out to be an issue on production
the chrome extension is loading iframe from staging.portal.forbole.com, which has different origin than x.forbole.com. as they are communicating using BroadcastChannel, it wont work with cross origin
i changed the iframe url on extension to be x.forbole.com as a fix. but in long term i will rewrite this to use another way for communication between extension and web since forbole x will be connected by different dapps in the future
please test with this extension
https://github.com/forbole/forbole-x/releases/download/v0.4.6/chrome-extension-v1.0.3.crx

I will ask the infra team to rewrite everything from staging.portal.forbole.com to x.forbole.com for now.


1.KSPA Praha

Basic information about the 4-year daily studium of IT technologies:
https://www.1kspa-praha.cz/index.php/cs/obory/18-it1#podrobnosti
IT technologies 4-year curriculum:
https://www.1kspa-praha.cz/images/stories/uceb-plany/IT 4lete.pdf
Sorry, everything is in Czech only.
Please note that while the school domain is 1kspa.cz, student email addresses are on the prahakspa.onmicrosoft.com, because we use Microsoft tools in the school.

@vosahloj Please provide us with a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students.
You can provide us with any of the following:

A copy of instruction (a hard-copy or electronic one) given to you by the school that tells you to use an email address in this email domain
An URL of a page at the official school website that says that students use this email domain
A reference to a recognized journal where an article was published by someone from the school, with explicit affiliation to the school, and e-mail address from the email domain
A WHOIS data that link the email domain to the school somehow
A copy of official letter from the school representative saying that the school provides e-mail addresses in this domain to the students
Anything else that can prove this fact in similar way, i.e. can be verified independently.

@vosahloj You might miss my previous comment. Please react.

@vosahloj I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for your understanding!


can not download the binary file at website

Describe the bug
when I go to the official website of Lens app, I can not download the binary.
To Reproduce
Steps to reproduce the behavior:

Go to https://k8slens.dev/
Click on 'Download Lens Desktop'
See error: "no healthy upstream"

Expected behavior
I should download the pre-built binary file.
Screenshots

Environment (please complete the following information):

Lens Version: Latest
OS: OSX
Installation method (e.g. snap or AppImage in Linux): download from website

Logs:
No logs
Kubeconfig:
Additional context

There was an outage to Lens backends this morning between 9-12am UTC. We apologise for the inconvenience this may have caused. Download should be available again now!


Failure in Rewriting ft.sh

The end of the "Creating a command-line script with arguments", the lesson is supposed to end with no failures from pytest. However, I still receive the following failure with the given code:
FAILED filetransfer_test.py::test_required_args - SystemExit: 2
This leaves 1 failed, 4 passed instead of the required 5 passed at the end of the lesson, which causes problems going into the next lesson.

For "Manipulating file paths", there are a number of issues with passing and failing in accordance with the instructions. Generally, it may be helpful that after every piece of code added, it would be good to have the correct number of passes and fails. Saying something like "Run pytest and there should be a new fail" can be misleading when the failures are not the correct failures. I found several issues with expected failures occur in the "Making sure paths exist" section. It might also be helpful to note that all testing was done with the failure from the previous comment still being an issue, so some problems could potentially be attributed to that.

Testing for a bad carrier path: I have gotten one failure, but it is simply because capsys was not added as a parameter despite using in the function. Once capsys is added, there is no longer a failure on this function. However, the previous error for test_required_args is still present during this test so there is still 1 failed, just in an unexpected place. In the next section--Validating carrier path--it states that this failure should be argparse.ArgumentError: argument --source: path/to/source does not exist.
Testing for available and writable destinations: For this whole section, the expected failures do not seem to be the errors actually received. At the point where there should be 3 failures, the 3 received from this section were:
FAILED filetransfer_test.py::test_missing_args - AssertionError: assert 'error: the following arguments are required' in 'usage: filetransfer.py [-h] --source SOURCE --id ID [--dest DEST]\...
FAILED filetransfer_test.py::test_non_existent_destination_path - AssertionError: assert 'path/to/dest does not exist' in 'usage: filetransfer.py [-h] --source SOURCE --id ID [--dest DEST]\nfiletransfer.py...
FAILED filetransfer_test.py::test_non_writable_destination_path - NameError: name 'bad_path' is not defined
When taking all of the following steps to fix the failures, it only fixes the first failure. I've tried to put in a real path directory in place of 'path/to/dest' but received the same failures still.

Since these were still failing at the end of the section, I did not go forward with "Creating the destination directory" since it continues to build on this.


Added more news

👷 Deploy Preview for balkantransport processing.
🔨 Explore the source changes: 41c944cee2b97ff5c6de0b9462bc1164ea7556e2
🔍 Inspect the deploy log: https://app.netlify.com/sites/balkantransport/deploys/612ea17b006a130008947bfa


[Maintenance] Add TwigToPdfGenerator

To keep both InvoicingPlugin and RefundPlugin consistent on how PDF is generated, in this PR, I did a similar refactor to this one: https://github.com/Sylius/RefundPlugin/pull/362.

Thanks, Kevin! :1st_place_medal:


Support new transactions and accounts structure

It closes #936
Still in the testing phase and most likely will need some tweaks.
Some parts are still in the process, but it is hard to implement it without ability to test.

How to test
Follow the instructions in README.md and then follow some extra steps:

Build go-spacemesh from the same commit that we're using now in the private testnet (now we're using
Copy it into smapp/node/YOUR_OS (mac / win / linux)
Make them executable chmod 755 ./node/mac/*
Create a new directory somewhere and put it there

networks.json (see below)
config.json which content you should take from here (if you have access): https://github.com/spacemeshos/Applications/blob/main/spacemesh-networks/devnet/999/config.yaml

Serve them (I am using python -m SimpleHTTPServer)
Run Smapp with dev env: DISCOVERY_URL=http://localhost:8000/networks.json yarn start
Create/open your wallet and choose the network from the list

networks.json
[
  {
    "netName": "PRIVATE TESTNET",
    "netID": 999,
    "conf": "http://<IP_ADDRESS>:8000/config.json",
    "grpcAPI": "https://api-devnet999.spacemesh.dev/",
    "jsonAPI": "https://api-json-devnet999.spacemesh.dev/",
    "explorer": "https://explorer.spacemesh.io/",
    "explorerAPI": "https://explorer-api-devnet225.spacemesh.io/",
    "explorerVersion": "v0.0.10",
    "explorerConf": "https://storage.googleapis.com/spacecraft-data/devnet225-archive/config.json",
    "dash": "https://dash.spacemesh.io/",
    "dashAPI": "wss://dash-api-devnet225.spacemesh.io/ws",
    "dashVersion": "v0.0.2",
    "repository": "spacemeshos/go-spacemesh-dev",
    "minNodeVersion": "d298a63",
    "maxNodeVersion": "d298a63",
    "minSmappRelease": "0.2.6",
    "latestSmappRelease": "0.2.6",
    "smappBaseDownloadUrl": "https://downloads.spacemesh.io",
    "nodeBaseDownloadUrl": "https://downloads.spacemesh.io"
  }
]

Don't be confused with most of the values here, the only things that we really need for testing are netID, conf, grpcAPI

To simplify testing transactions I also wrote a small CLI-tool that composes and optionally signs the transaction. Afterward you can just copy the byte array of the signed transaction and send it to TransactionService.SubmitTransaction GRPC API via your favorite GRPC-client (E.G. grpcurl).
Just run yarn debug:tx and follow instructions ;)

Since it was already approved and I made the required changes — I'm merging it.
@maparr if you find something else — feel free to leave some comments there and mention me, I'll add these fixes then.


Test: Do not gather envoy.log

Due adeacc11586b2529385924a72702d102fa8fe20b gather envoy log is not
longer needed.
Signed-off-by: Eloy Coto<EMAIL_ADDRESS>

test-me-please


Stereo information lost

Hi,
I am using the following C++ script to read a molecule from a SD file (with defined stereochemistry), but the information about the stereochemistry gets lost. The file I am using is from glucose (ZINC03833800; http://zinc.docking.org/substance/3833800)
When I check the chiral information of the atoms, all of them have the chiral tag 0.
How can I keep the stereo information?
SCRIPT:
std::fstream infilestr;
infilestr.open(in_file_name.c_str( ));

RDKit::SDMolSupplier mol_supplier( in_file_name , true );

for (unsigned i = 0; i < mol_supplier.length(); ++i) {
        RDKit::ROMOL_SPTR mol(mol_supplier[i]);

        if (!mol){
            std::cout << "Error, molecule not found!" << std::endl;
            continue;
        }

        for (unsigned int j = 0; j < mol->getNumAtoms(); j++){
            RDKit::Atom* atom = mol->getAtomWithIdx(j);
            std::cout << "Chirality atom " << j+1  << ": " << atom->getChiralTag() << std::endl;
        }

        std::string usmiles = RDKit::MolToSmiles (*mol);
        std::cout << "Mol: " << usmiles << std::endl;
}

OUTPUT FOR GLUCOSE (ZINC03833800):
Chirality atom 1: 0
Chirality atom 2: 0
Chirality atom 3: 0
Chirality atom 4: 0
Chirality atom 5: 0
Chirality atom 6: 0
Chirality atom 7: 0
Chirality atom 8: 0
Chirality atom 9: 0
Chirality atom 10: 0
Chirality atom 11: 0
Chirality atom 12: 0
Mol: OCC1OC(O)C(O)C(O)C1O

There are a couple of things going on here.
Currently if you want the RDKit to figure out the stereochemistry from the 3D coordinates, you need to make a call to MolOps::assignStereochemistryFrom3D() after reading in the molecule.
If you do this, the individual atoms should have chirality present.
If you want to see that chirality in the SMILES, you need to set the optional second argument to MolToSmiles: std::string usmiles = RDKit::MolToSmiles(*mol,true)
These defaults are set as they are for historical reasons and we're in the process of changing them as part of #1679.

Thanks for your answer!
But I would need to read the stereochemistry information from the SD file and not calculate the chirality from the coordinates, since this way the information about racemates would get lost.
Is there a way to do that?