When I ran %sentenceTokenizer1 with the sample news table which contains 600 news article, the applyConcept action ran indefinitely. This is because the part of the rule section below does not use the right operators:

PREDICATE_RULE:SentBoundaries(first,last):(SENT,"_first{_w}","_last{_w}")

In order to find the first word in a sentence, you must use the SENTSTART_n operator with n=1, and the same goes with the last word in the sentence (using the SENTEND_n operator):

(SENT, (SENTSTART_1, "_first{_w}"), (SENTEND_1, "_last{_w}"))

Using the code below, it took less than 2 seconds to run applyConcept action on Viya 3.4( with CAS in SMP):

cas;
caslib _all_ assign;
libname mylib "/home/sasdemo/data";
libname sascas1 cas caslib=casuser;

data casuser.news;
set mylib.news;
_document_ = _n_;
run;

%let language=English;
%let dsIn=news;
%let docVar=_document_;
%let textVar=text;
%let language=English;
%let dsOut=sentences1;

/* Rule for determining sentence boundaries */
data sascas1.concept_rule;
length rule $ 200;
ruleId=1;
rule='ENABLE:SentBoundaries';
output;
ruleId=2;
/* rule='PREDICATE_RULE:SentBoundaries(first,last):(SENT,"_first{_w}","_last{_w}")'; */
rule='PREDICATE_RULE:SentBoundaries(first,last):(SENT, (SENTSTART_1, "_first{_w}"), (SENTEND_1, "_last{_w}"))';
output;
run;

proc cas;
textRuleDevelop.validateConcept /
table={name="concept_rule"}
config='rule'
ruleId='ruleId'
language="&language"
casOut={name='outValidation',replace=TRUE}
;
run;
quit;

/* Compile concept rule; */
proc cas;
textRuleDevelop.compileConcept /
table={name="concept_rule"}
config="rule"
enablePredefined=false
language="&language"
casOut={name="outli", replace=TRUE}
;
run;
quit;

/* Get Sentences */
proc cas;
textRuleScore.applyConcept /
table={name="&dsIn"}
docId="&docVar"
text="&textVar"
language="&language"
model={name="outli"}
matchType="best"
casOut={name="outpos_eli", replace=TRUE}
factOut={name="&dsOut", replace=TRUE, where="_fact_argument_=''"}
;
run;
quit;

The following is the log from running the applyConcept action:

134 proc cas;
135 textRuleScore.applyConcept /
136 table={name="news_noMiss"}
137 docId="_document_"
138 text="text"
139 language="English"
140 model={name="outli2"}
141 matchType="best"
142 casOut={name="outpos_eli", replace=TRUE}
143 factOut={name="sentence_out2", replace=TRUE /*, where="_fact_argument_=''" */}
144 ;
145 run;
NOTE: Active Session now CASAUTO.

146 quit;

NOTE: PROCEDURE CAS used (Total process time):
real time 1.60 seconds
cpu time 0.05 seconds

In the Results, the rule is validated without errors. Also, the factout= (sentence1) table contains 6767 rows. Note that the casout= (outpos_eli) table contains no rows. This is expected as you should only get a fact table and no concept table with this rule.

Hope this helps!

Ann